Download - CS 387/680: GAME AI
![Page 1: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/1.jpg)
CS 387/680: GAME AI DECISION MAKING
4/21/2014 Instructor: Santiago Ontañón
[email protected] TA: Alberto Uriarte
office hours: Tuesday 4-6pm, Cyber Learning Center Class website:
https://www.cs.drexel.edu/~santi/teaching/2014/CS387-680/intro.html
![Page 2: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/2.jpg)
Reminders • Check BBVista site for the course regularly • Also: https://www.cs.drexel.edu/~santi/teaching/2014/CS387-680/intro.html
• Deadline for people doing Project 1 today!. • Submission available via learn.drexel.edu • Any questions?
• Deadline for project 2 next week. • Any questions?
![Page 3: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/3.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 4: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/4.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 5: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/5.jpg)
Decision Making • So far we have seen:
• Movement (steering behaviors) • Pathfinding (A* variants)
• We assumed characters knew where to go (need to know destination to call A*).
• This week: • How do characters decide what to do? Where to go? • Chapter 5 of the textbook
![Page 6: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/6.jpg)
Decision Making • A situation is characterized by:
• Known information about the state of the world • Unknown information about the state of the world • Set of possible actions to execute
• Problem: • Given a situation, which of the
possible actions is the best?
![Page 7: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/7.jpg)
Example: HL2 • Known information:
• Position, map, strider position, time
• Unknown: • (nothing, game AI
cheats J)
• Actions: • Turn towards X • Walk to (X,Y) • Run to (X,Y) • Fire at X • Play animation X
![Page 8: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/8.jpg)
Example: RTS Games • Known information:
• Player data, explored terrain
• Unknown: • Unexplored terrain • Enemy strategy
• Actions: • Build barracks • Build refinery • Build supply depot • Wait • Explore • etc.
![Page 9: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/9.jpg)
Example: Final Fantasy VI • Known information:
• Party information • Two enemies
• Unknown: • Resistances • Attack power • Remaining health
• Actions:
![Page 10: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/10.jpg)
Game AI Architecture
AI
World Interface
(perception)
Strategy
Decision Making
Movement
![Page 11: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/11.jpg)
Game AI Architecture
AI
World Interface
(perception)
Strategy
Decision Making
Movement
Scripting techniques covered this week applicable to both
“Strategy” and “Decision Making” levels of game AI
![Page 12: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/12.jpg)
Game AI Architecture
AI
World Interface
(perception)
Strategy
Decision Making
Movement
The distinction will be: 1) Strategy involves many
characters 2) Decision Making involves a
single character
![Page 13: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/13.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 14: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/14.jpg)
Scripting • Most Game AI is scripted
• Game Engines allow game developers to specify the behavior of characters in some scripting language: • Lua • Scheme • Python • Etc. • Game specific languages
![Page 15: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/15.jpg)
Game AI Architecture: Scripting
AI
World Interface
(perception)
Strategy
Decision Making
Movement
![Page 16: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/16.jpg)
Game AI Architecture: Scripting AI
World Interface
(perception)
Strategy
Tick(GameState *gs,int time) {!!If (time>=100 &&!! ! notRunning()!
startRunning(BUILDING4);!}!
Movement
![Page 17: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/17.jpg)
Scripting • Advantages:
• Easy • Gives control to the game designers (characters do whatever the
game designers want)
• Disadvantages: • Labor intensive: the behavior of each characters needs to be
predefined for all situations • Rigid:
• If the player finds a hole in one of the behaviors, she can exploit it again and again
• Not adaptive to what the player does (unless scripted coded to be so)
![Page 18: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/18.jpg)
Adding Scripting to your Game Engine • Your Game Engine will have classes for things like:
• Game Objects: • Characters • Enemies • Items • Spells • etc.
• Maps • Game
• Add the capability of adding scripts to all of them.
![Page 19: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/19.jpg)
Adding Scripting to your Game Engine • When you add a script to a class, typically you want four
functionalities: • Script that runs “on wakeup” • Script that runs “on each frame” • Script that runs “on disposal” • Script that runs “on certain other events”
• Each script is a function that runs when the appropriate event happens, triggers whichever actions the game designer desires and then terminates (scripts might have internal state, so that subsequent executions can use results from previous executions).
![Page 20: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/20.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 21: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/21.jpg)
Finite State Machines • Scripts allow for a lot of flexibility, but they have one
problem…
![Page 22: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/22.jpg)
Finite State Machines • Scripts allow for a lot of flexibility, but they have one
problem… • They require the game designer to have programming knowledge • They are too powerful (often dangerous)
• Graphical approaches have been designed for easy AI authoring: • The most common are finite state machines
![Page 23: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/23.jpg)
Finite State Machines
Harvest Minerals
Build Barracks
Train marines
Attack Enemy Explore
Train SCVs
4 SCVs harvesting
4 SCVs Less than
4 SCVs
barracks
4 marines &
Enemy seen
Enemy seen
4 marines & Enemy unseen No
marines
![Page 24: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/24.jpg)
Finite State Machines
Harvest Minerals
Build Barracks
Train marines
Attack Enemy Explore
Train SCVs
4 SCVs harvesting
4 SCVs Less than
4 SCVs
barracks
4 marines &
Enemy seen
Enemy seen
4 marines & Enemy unseen No
marines
Many graphical tools exist, But if you wanted to program this by hand, How would you translate this to code?
![Page 25: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/25.jpg)
Finite State Machines • Easy to implement:
switch(state) { case START: if (numSCVs<4) state = TRAIN_SCVs;
if (numHarvestingSCVs>=4) state = BUILD_BARRACKS; Unit *SCV = findIdleSCV(); Unit *mineral = findClosestMineral(SCV); SCV->harvest(mineral); break;
case TRAIN_SCVs: if (numSCVs>=4) state = START; Unit *base = findIdleBase(); base->train(UnitType::SCV); break;
case BUILD_BARRACKS: … }
![Page 26: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/26.jpg)
Finite State Machines
• Good for simple AIs • Become unmanageable for complex tasks • Hard to maintain
![Page 27: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/27.jpg)
Finite State Machines (Add a new state)
Harvest Minerals
Build Barracks
Train marines
Attack Enemy Explore
Train SCVs
4 SCVs harvesting
4 SCVs Less than
4 SCVs
barracks
4 marines &
Enemy seen
Enemy seen
4 marines & Enemy unseen No
marines
How would you change this FSM to “attach an enemy if it comes into our base”?
![Page 28: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/28.jpg)
Finite State Machines (Add a new state)
Harvest Minerals
Build Barracks
Train 4 marines
Attack Enemy Explore
Train 4 SCVs
4 SCVs harvesting
4 SCVs Less than
4 SCVs
barracks
4 marines &
Enemy seen
Enemy seen
Enemy unseen
No marines
Attack Inside Enemy
Enemy Inside Base
![Page 29: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/29.jpg)
Standard Strategy
Hierarchical Finite State Machines
Attack Inside Enemy
Enemy Inside Base
No Enemy Inside Base
• FSM inside of the state of another FSM • As many levels as needed • Can alleviate complexity problem to some extent
![Page 30: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/30.jpg)
Hierarchical Finite State Machines
Harvest Minerals
Build Barracks
Train 4 marines
Attack Enemy Explore
Train 4 SCVs
4 SCVs harvesting
4 SCVs Less than
4 SCVs
barracks
4 marines &
Enemy seen
Enemy seen
Enemy unseen
No marines
Attack Inside Enemy
Enemy Inside Base
No Enemy Inside Base
• FSM inside of the state of another FSM • As many levels as needed • Can alleviate complexity problem to some extent
![Page 31: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/31.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 32: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/32.jpg)
Decision Trees • In the FSM examples before, decisions were quite simple:
• If “4 SCVs” then “build barracks”
• But those conditions can easily become complex
• Decision trees offer a way to encode complex decisions in a easy and organized way
![Page 33: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/33.jpg)
Example of Complex Decision • Decide when to attack the enemy in a RTS game, and what
kind of units to build
• We could try to define a set of rules: • If we have not seen the enemy then build ground units • If we have seen the enemy and he has no air units and we have more
units than him, then attack • If we have seen the enemy and he has air units and we do not have air
units, then build air units • etc.
• Problems: • complex to know if we are missing any scenario, • The conditions of the rules might grow very complex
![Page 34: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/34.jpg)
Example of Complex Decision: Decision Tree
Enemy Seen
Does he have Air Units
Do we have Antiair units
Do we have More units Than him
Do we have More units Than him
Build Antiair units
Attack!
Build more units
Attack!
Build more units
Build more units
no no
no no
no
yes
yes
yes
yes
yes
• The same decision, can be easily captured in a decision tree
![Page 35: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/35.jpg)
Decision Trees • Intuitive
• Help us determine whether we are forgetting a case
• Easy to implement: • Decision trees can be used as “paper and pencil” technique, to
think about the problem, and then just use nested if-then-else statements
• They can also be implemented in a generic way, and give graphical editors to game designers
![Page 36: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/36.jpg)
Finite State Machines with Decision Trees • In complex FSMs, conditions in arches might get complex
• Each state could have a decision tree to determine which state to go next
S1 C1 C2
S2
S3
![Page 37: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/37.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 38: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/38.jpg)
Behavior Trees • Combination of techniques:
• Hierarchical state machines • Scheduling • Reactive planning • Action Execution
• Increasingly popular in commercial games
• Strength: • Visual and easy to understand way to author behaviors and
decisions for characters without having programming knowledge
![Page 39: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/39.jpg)
Behavior Trees Appeared in Halo 2
Halo 2 (2004)
![Page 40: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/40.jpg)
Example Behavior Tree
![Page 41: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/41.jpg)
Behavior Tree Basics • A behavior tree (BT) captures the “behavior” or “decision
mechanism” of a character in a game
• At each frame (if synchronous): • The game engine executes one cycle of the BT: • As a side effect of execution, a BT executes actions (that control
the character)
• The basic component of a behavior tree is a task
![Page 42: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/42.jpg)
Tasks • Each “node” in a behavior tree is a task
• Tasks have one thing in common: • At each game cycle, a cycle of a task is executed • It returns success, failure, error, etc. • As a side effect they might execute things in the game
• Three basic types of tasks: • Conditions • Actions • Composites
![Page 43: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/43.jpg)
Tasks: Conditions • Test some Boolean property of the game
• Example: • Test of proximity (any door nearby? Is the player nearby?) • Collision test (e.g. collision ray)
• Condition tasks are typically parametrizable so they can be reused
• Upon execution they just perform the test and return “succeed” or “fail” depending on the result of the test.
![Page 44: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/44.jpg)
Tasks: Actions • Actions alter the state of the game
• Examples: • Trigger an animation • Play a sound • Update internal state of character (e.g. ammo, health, position) • Trigger path-finding
• Typically actions succeed, so they tend to return always “succeed”
![Page 45: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/45.jpg)
Tasks: Composites • Actions and conditions are the leaves of a BT • Composites are the internal nodes and define ways to
combine tasks
• Examples: • Sequence: executes all its children in sequence, if any fail, return
failure, if all succeed, return success • Selector: executes all its children in sequence until one succeeds,
then return success. If all of them fail, return failure.
• There are other types of selector tasks (as we will see later)
![Page 46: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/46.jpg)
Behavior Tree Tasks
Action Condition Sequence Selector
![Page 47: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/47.jpg)
Behavior Tree Tasks
Action Condition Sequence Selector
Generic: the same for all games
Domain dependent: Each game defines its own actions and conditions
![Page 48: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/48.jpg)
Simplest Behavior Tree • Goal: Make a character move right
Move Right
![Page 49: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/49.jpg)
Example Execution: • Goal: Make a character move right
Move Right
![Page 50: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/50.jpg)
Example Execution: • Goal: Make a character move right
Move Right
![Page 51: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/51.jpg)
Simplest Behavior Tree • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 52: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/52.jpg)
Example Execution: • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 53: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/53.jpg)
Example Execution: • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 54: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/54.jpg)
Example Execution: • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 55: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/55.jpg)
Example Execution: • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 56: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/56.jpg)
Example Execution: • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 57: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/57.jpg)
Example Execution: • Goal: Make a character move right when the player is
near
Move Right
Is Player Near?
Sequence
![Page 58: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/58.jpg)
What If There Are Obstacles? • Goal: Make a character move right when the player is
near, even if the door is closed
Open Door
Is Player Near?
Is Door Closed?
Is Door Open?
Move Right
Move to Door
![Page 59: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/59.jpg)
What If There Are Obstacles? • Goal: Make a character move right when the player is
near, even if the door is closed
Open Door
Is Player Near?
Is Door Closed?
Move Right
Is Door Open?
Move Right
Move to Door
Sequence
Selector
Sequence Sequence
![Page 60: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/60.jpg)
Behavior Tree GUIs • Behavior trees largest strength is that they are visual
• Coupled with a graphical editor, they allow game designers to create character AI without programming
• They are easy to edit, and can be deployed in the game at any stage, always producing executable code
![Page 61: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/61.jpg)
Behavior Tree GUIs
![Page 62: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/62.jpg)
Behavior Tree GUIs
![Page 63: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/63.jpg)
Implementation of Behavior Trees • All nodes in the tree extend one basic class: Task
class Task { public: bool run(GameState gs, Character c);
};
![Page 64: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/64.jpg)
Implementation of Action Tasks class MoveRight public Task {
public: bool run(GameState gs, Character c) {
c.setTarget(c.getX()+10,c.getZ()); return true;
}
};
![Page 65: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/65.jpg)
Implementation of Action Tasks class MoveRight2 public Task {
public: bool run(GameState gs, Character c) {
if (gs.existsPath(c.getPos(),c.getPos()+new Vector(10,0,0)) { c.setTarget(c.getX()+10,c.getZ()); return true; } else { return false; }
} };
![Page 66: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/66.jpg)
Implementation of Condition Tasks class IsPlayerNear public Task {
public: bool run(GameState gs, Character c) {
Character player = gs.getPlayer(); return (player.distance(c) < 10);
}
};
![Page 67: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/67.jpg)
Implementation of Composite Tasks Class Composite public Task {
public: Composite(list<task> &a_children); protected: list<task> m_children;
};
![Page 68: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/68.jpg)
Implementation of Composite Tasks: Sequence Class Sequence public Composite {
public: Sequence(list<Task> &a_children) : Composite(a_children) {
current = m_children.begin(); } bool run(GameState gs, Character c) {
if (current == m_children.end()) return true; if (*current.run(gs,c)) { current++; return true; } else { return false; }
} protected: list<Task>::iterator current;
};
![Page 69: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/69.jpg)
Implementation of Composite Tasks: Sequence Class Sequence public Composite {
public: Sequence(list<Task> &a_children) : Composite(a_children) {
current = m_children.begin(); } bool run(GameState gs, Character c) {
if (current == m_children.end()) return true; if (*current.run(gs,c)) { current++; return true; } else { return false; }
} protected: list<Task>::iterator current;
};
An alternative is to define the “run” function as returning an enum/integer,
and here return “need_more_time”
![Page 70: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/70.jpg)
Implementation of Composite Tasks: Sequence (If BT runs in a separate thread) Class Sequence public Composite {
public: Sequence(list<Task> &a_children) : Composite(a_children) { } bool run(GameState gs, Character c) {
for(list<Task>::iterator current = m_children.begin(); current!=m_children.end(); current++) { if (!*current.run(gs,c)) return false; } return true;
} };
![Page 71: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/71.jpg)
Additional Useful Composites • Like a Selector, but randomizes the
order in which children are checked
• Like a Sequence, but randomizes the order in which the children are executed
• Runs all the children in parallel (if one fails, it fails)
RandomSelector
RandomSequence
Parallel
![Page 72: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/72.jpg)
Implementing Random Selector/Sequence Class RandomSelector public Composite {
public: RandomSelector (list<Task> &a_children) : Composite(a_children) { } bool run(GameState gs, Character c) {
while(true) { Task *child = m_children.getRandom(); if (*child.run(gs,c)) return true; }
} };
![Page 73: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/73.jpg)
Implementing Random Selector/Sequence Class RandomSelector public Composite {
public: RandomSelector (list<Task> &a_children) : Composite(a_children) { } bool run(GameState gs, Character c) {
m_children.shuffle(); for(list<Task>::iterator current = m_children.begin(); current!=m_children.end(); current++) { if (!*current.run(gs,c)) return false; } return true;
} };
![Page 74: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/74.jpg)
Decorators • Advanced Behavior Tree editors/engines allow for a 4th
type of node: called a decorator
• The concept of a decorator comes from OOP: • An object with the same interface as another one, and that modifies
it in some way • External objects do not have to know if they are dealing with the
original object or with the decorator
• In the context of BTs, a decorator is like a composite but with a single child (they could be considered as a special case of composites)
![Page 75: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/75.jpg)
Useful Decorators • Repeats the child node N times
• Repeats the child until it fails
• Executes the child only if some condition is met
• executes child and returns the opposite value
• Used to guard some resource that can only be used once
Repeat(N)
UntilFail
Filter
Inverter
Semaphore(P)
![Page 76: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/76.jpg)
Data in Behavior Trees • Recall the example BT we saw at the beginning of the
class:
Move Right
Is Player Near?
Sequence
![Page 77: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/77.jpg)
Data in Behavior Trees • Recall the example BT we saw at the beginning of the
class:
Move Right
Is Player Near?
Sequence Now imagine we want it to move TOWARDS the
player
![Page 78: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/78.jpg)
Data in Behavior Trees • Recall the example BT we saw at the beginning of the
class:
Move to
Player
Is Player Near?
Sequence We can have a special action to do
so:
![Page 79: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/79.jpg)
Data in Behavior Trees • Recall the example BT we saw at the beginning of the
class:
Move to
Player
Is Player Near?
Sequence Now, what if there
are multiple players and we want to move to
one of them?
![Page 80: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/80.jpg)
Data in Behavior Trees • Behavior Trees are coupled with a blackboard
• A blackboard is a global structure where we can store and read data by name (there is no concept of “context”)
Move to
Player
Is Player Near?
Sequence Blackboard:
![Page 81: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/81.jpg)
Data in Behavior Trees • Behavior Trees are coupled with a blackboard
• A blackboard is a global structure where we can store and read data by name (there is no concept of “context”)
Move to
Player
Is Player Near?
Sequence Blackboard:
Player: GO-0034
![Page 82: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/82.jpg)
Data in Behavior Trees • Behavior Trees are coupled with a blackboard
• A blackboard is a global structure where we can store and read data by name (there is no concept of “context”)
Move to
Player
Is Player Near?
Sequence Blackboard:
Player: GO-0034
![Page 83: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/83.jpg)
Blackboards • Blackboards are common in artificial intelligence (not just
Game AI)
• Useful to coordinate multiple agents: • In the case of a BT, coordinating the different tasks of the tree
• The blackboard can store information such as: • Current target • Last safe position seen • etc.
• In general, all the dynamic information that is not stored in the character or in the game state
![Page 84: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/84.jpg)
Implementation of a Blackboard class Blackboard { public:
void put(String key, Object *value) { store.put(key, value);
} Object *get(String key) {
Return store[key]; }
protected:
map<string,Object *> store; };
![Page 85: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/85.jpg)
The Behavior Tree Pipeline • 1) Define your Tasks (Actions, Conditions, Composites
and Decorators)
• 2) Author the BTs for each of your characters: • You can use a graphical editor • Or directly author the BTs in a text file • Or directly create a class that encapsulates your BT
• 3) Each time a character is spawned in the game, instantiate a BT for it
![Page 86: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/86.jpg)
The Behavior Tree Pipeline • 1) Define your Tasks (Actions, Conditions, Composites
and Decorators)
• 2) Author the BTs for each of your characters: • You can use a graphical editor • Or directly author the BTs in a text file • Or directly create a class that encapsulates your BT
• 3) Each time a character is spawned in the game, instantiate a BT for it
Either of these two would be
good for Project 3
![Page 87: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/87.jpg)
Authoring BTs in a Text File • Example:
// My behavior tree definition: <sequence>
<condition>IsPlayerNear</condition> <action>MoveToPlayer</action>
</sequence> Move to
Player
Is Player Near?
Sequence
![Page 88: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/88.jpg)
Using a class to encapsulate a BT • Example:
class EnemyBT1 { public: EnemyBT1() {
list<Task> c; c.pushBack(new IsPlayerNear()); c.pushBack(new MoveToPlayer()); head = new Sequence(c);
} private: Task *head; };
Move to
Player
Is Player Near?
Sequence
![Page 89: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/89.jpg)
Behavior Tree Reuse • Whole Behavior Trees encapsulated as Tasks
• In that way, we can author sub-trees, and then reuse them in other BTs, as if they were subroutines
• Assign BTs a goal: • Give each BT a goal (a goal is just a string, like “attack player”) • Author several BTs for each goal • Define a SubGoal task simply looks up for a tree with the given
goal and executes it. SubGoal tasks can be implemented as “selectors” (selecting the different sub-trees until one works)
• This can give variety to the behavior of characters
![Page 90: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/90.jpg)
Limitations of Behavior Trees • BTs excel at ease of authoring for non-programmers, and
are common in modern video games. But they have limitations
• The biggest limitation of BTs is that it is hard to author FSM-like behaviors: • BTs are natural for reactive behavior • If we need a character that changes states depending on events,
BTs become cumbersome (it can be done, but it is cumbersome)
![Page 91: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/91.jpg)
Outline • Decision Making • Scripting Languages • Finite State Machines • Decision Trees • Behavior Trees • Decision Theory
![Page 92: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/92.jpg)
Authoring AI vs Autonomous AI • FSMs, Decision Trees, Rule-based Systems, etc. are
useful to hardcode decisions: • Game designer tools to make the AI behave the way they want • The AI will never do anything the game designers didn’t foresee
(except for bugs)
• We will now change our attention to techniques that can
be used to let the AI autonomously take decisions • The AI takes decisions on its own, and can generate strategies, not
foreseen by game designers
![Page 93: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/93.jpg)
Decision Theory • Given a situation, decide which action to perform
depending on the desirability of its immediate outcome
• Desirability of a situation: utility function U(s) • Decision theory is based upon the idea that there is such utility
function
• Example utility function, Chess: • Score of a player: 10 points for the queen + 5 points per rook, + 3
points per knight or bishop + 1 point per pawn. • Utility for white pieces: Uw(s) = Score(white) – Score(black) • Utility for black pieces: Ub(s) = Score(black) – Score(white)
![Page 94: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/94.jpg)
Example Utility Function for RTS games: • Similarly to chess, we can do:
• Enemy units must be estimated
• This is oversimplified, but it is good as a first approach. It can be improved by adding: • Resources: (minerals, gas, gold, wood, etc.) • Research done • Territory under control • etc.
U(s) =X
t
wt(cfriendlyt � cenemy
t )
![Page 95: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/95.jpg)
Is it Realistic to Have a Utility Function? • Yes:
• In hardcoded approaches (FSM, Decision trees, etc.): • The AI designer has to tell the AI HOW to play
• Utility function: • Only captures the goal of the game • In the simplest form, utility is simply 1 for win, -1 for loss, and 0
otherwise • But the more information conveyed in the utility function, the better the
AI can decide what to do • The AI designer has to tell the AI WHAT is the goal
• It is sometimes easier to define a utility function than to hardcode a strategy, since utility function has less information (only WHAT, not HOW)
![Page 96: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/96.jpg)
Decision Theory • Effect of an action a on the state s: Result(s,a)
• Since we might not know the exact state in which we are, we can only estimate the result of an action: • P(Result(s,a) = s’ | e) (e is the information we know about s)
• Example: • a = “attack enemy supply depot with 4 marines” • e = “we haven’t observed any enemy unit around the supply depot” • Result(s,a) = “supply depot destroyed in 250 cycles, 4 marines intact” • But we don’t know if there were cloaked units, so we can only guess
the result of a
![Page 97: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/97.jpg)
Maximum Expected Utility Principle (MEU) • Select the action with the expected maximum utility:
• Requires: • Utility function (hardcoded or machine learned) • Estimation of action effects (hardcoded or machine learned)
• The AI has to know what the actions do!
EU(a|e) =X
s0
P (Result(a, s) = s0|e)U(s0)
![Page 98: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/98.jpg)
Example: Target Selection • Utility function:
• 60 points per footman • 400 points per barracks • 200 points per lumber mill
• Which action? • Attack enemy footman • Attack enemy barracks • Attack enemy lumber mill
Player = blue Enemy = red
![Page 99: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/99.jpg)
Example: Target Selection • Utility function:
• 60 points per footman • 400 points per barracks • 200 points per lumber mill
• Which action? • Attack enemy footman:
• 2 footmen can kill 1 footman
• U(s’) = 2*60 – 400 – 200 = -480 Player = blue Enemy = red
![Page 100: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/100.jpg)
Example: Target Selection • Utility function:
• 60 points per footman • 400 points per barracks • 200 points per lumber mill
• Which action? • Attack enemy barracks:
• During the time it takes to destroy the barracks, the enemy footman can kill our 2 footmen
• U(s’) = -60 – 400 – 200 = -660 Player = blue Enemy = red
![Page 101: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/101.jpg)
Example: Target Selection • Utility function:
• 60 points per footman • 400 points per barracks • 200 points per lumber mill
• Which action? • Attack enemy lumber mill:
• During the time it takes to destroy the lumber mill, the enemy footman can kill our 2 footmen
• U(s’) = -60 – 400 – 200 = -660 Player = blue Enemy = red
![Page 102: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/102.jpg)
Example: Target Selection • Utility function:
• 60 points per footman • 400 points per barracks • 200 points per lumber mill
• Which action? • Attack enemy footman: -480 • Attack enemy barracks: -660 • Attack enemy lumber mill: -660
Player = blue Enemy = red
![Page 103: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/103.jpg)
Game AI Architecture
AI
World Interface
(perception)
Strategy
Decision Making
Movement
Decision Theory can be useful at these two levels.
![Page 104: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/104.jpg)
Value of Information • How do we know when is it worth spending resources in
exploring?
• Value of perfect information:
• i.e.:
• How much utility can we expect to gain if we knew the value of an unknown variable E (that can take k different values e1, …, ek)
VPI e(E) =
X
k
P (E = ek)EU (a⇤k|e, E = ek)
!� EU (a⇤|e)
![Page 105: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/105.jpg)
Example • Starcraft:
• Player force: 4 marines (60 points each) • 1 Enemy Command Center spotted (500 points) • Enemy defenses: unknown
• Which action to perform? • Attack • Train more marines
4 marines
command center
?
![Page 106: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/106.jpg)
Example • Starcraft:
• Player force: 4 marines (60 points each) • 1 Enemy Command Center spotted (500 points) • Enemy defenses: unknown
• Which action to perform? • Attack:
• U(s’) = 0.5 U(winning) + 0.5 U(losing) = • U(s’) = 0.5 * (4 marines) + 0.5 (-1 command center) = -230
• Train more marines: • U(s’) = 6 marines – (1 command center + 2 SCVs) = -380
4 marines
command center
?
![Page 107: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/107.jpg)
Example • Starcraft:
• Player force: 4 marines (60 points each) • 1 Enemy Command Center spotted (500 points) • Enemy defenses: unknown
• Which action to perform? • Attack:
• U(s’) = 0.5 * (4 marines) + 0.5 (-1 command center) = -130 • Train more marines:
• U(s’) = 6 marines – (1 command center + 2 SCVs) = -380
• If we know no defenses, then for Attack: • U(s’) = 4 marines = 240
• If we know there are defenses, then for Attack: • U(s’) = -1 command center = -500
4 marines
command center
?
EU(Attack | Defenses) = -500
EU(Attack | no Defenses) = 240
![Page 108: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/108.jpg)
Example • Starcraft:
• Player force: 4 marines (60 points each) • 1 Enemy Command Center spotted (500 points) • Enemy defenses: unknown
• Which action to perform? • Attack:
• U(s’) = 0.5 * (4 marines) + 0.5 (-1 command center) = -230 • Train more marines:
• U(s’) = 6 marines – (1 command center + 2 SCVs) = -380
VPI(defenses) = (0.5 * EU(More Marines | Defenses) +
0.5 * EU(Attack | No Defenses) ) – (EU(Attack))
4 marines
command center
?
![Page 109: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/109.jpg)
Example • Starcraft:
• Player force: 4 marines (60 points each) • 1 Enemy Command Center spotted (500 points) • Enemy defenses: unknown
• Which action to perform? • Attack:
• U(s’) = 0.5 * (4 marines) + 0.5 (-1 command center) = -230 • Train more marines:
• U(s’) = 6 marines – (1 command center + 2 SCVs) = -380
VPI(defenses) = (0.5 * -380 +
0.5 * 240 ) – (-130) = -70 + 130 = 60
4 marines
command center
?
![Page 110: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/110.jpg)
Decision Theory • Basic principles for making rational decisions
• Goal of game AI is to be fun: • Utility function doesn’t have to be tuned to “optimal play”, but to
“fun play” • For example, utility function may penalize too many attacks to the
player in a given period of time, in order to let the player breathe.
• Deals only with immediate utility. No look ahead, or adversarial planning: • To deal with that: adversarial search (week 8)
![Page 111: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/111.jpg)
Project 1: Steering Behaviors • Implement steering behaviors (explained today) • Game Engine:
• PTSP (Java)
• Competition: • http://www.ptsp-game.net/index.php • (if you are interested)
![Page 112: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/112.jpg)
Project 2: Pathfinding • Implement A* in a RTS Game • Game Engine: S3 (Java)
![Page 113: CS 387/680: GAME AI](https://reader031.vdocuments.us/reader031/viewer/2022011815/61d4e2d53fb8fe03d80c36cd/html5/thumbnails/113.jpg)
Project 3: Scripting • Implement a Behavior Tree
• Game Engine: • Mario (Java, with bindings
for other languages)
• Competition: • https://code.google.com/p/marioai/ • (if you are interested)