the next generation of game planners
DESCRIPTION
Tutorial given during the 2011 Paris Game AI Conference on Automated Planning applied to GamesTRANSCRIPT
The Next Generation of Game Planners
The "Everything You (N)Ever Wanted to Know" Tour
Luke DickenStrathclyde AI and Games Research Group
University of Strathclyde
Controversy!
2
Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
2
Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
• “Academia has long discarded such planners in favor of
hierarchical ones...”
2
Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
• “Academia has long discarded such planners in favor of
hierarchical ones...”
2
Alex, “This Year in Game AI”(Jan ’11)
Controversy!
• “...STRIPS-style goal oriented action planning has turned out
to be a dead end.”
• “Academia has long discarded such planners in favor of
hierarchical ones...”
• This session will drill into what Automated Planning is and
why (some) parts of it are still relevant for Game AI
2
Alex, “This Year in Game AI”(Jan ’11)
What is Automated Planning?
3
What is Automated Planning?
• “Strong” AI
3
What is Automated Planning?
• “Strong” AI
• Finds action sequences - Plan
3
What is Automated Planning?
• “Strong” AI
• Finds action sequences - Plan
• Over 40 years of research
3
What is Automated Planning?
• “Strong” AI
• Finds action sequences - Plan
• Over 40 years of research
• Planning Domain Description Language (PDDL) - 1998
3
How Does it Work?
4
How Does it Work?
1. Description of actions possible
4
How Does it Work?
1. Description of actions possible
2. Complete description of initial state of the world
4
How Does it Work?
1. Description of actions possible
2. Complete description of initial state of the world
3. Definition of goals that need to be achieved
4
Planning
5
S0
Planning
5
S0
S1 S2 S3
Planning
5
S0
S1
Planning
5
S0
S1
S4 S5 S6
Planning
5
S0
S1
S6
Planning
5
S0
S1
S6
And so on, until goal reached.
GOAP
6
GOAP
6
GOAP
6
GOAP
6
GOAP
6
GOAP
6
Issues with GOAP
7
Issues with GOAP
• Issue 1 : Lack of directorial control.
7
Issues with GOAP
• Issue 1 : Lack of directorial control.
‣When NPCs get smart enough to realise standing next
to exploding barrels is hazardous, cinematic experience is
diminished.
7
Issues with GOAP
• Issue 1 : Lack of directorial control.
‣When NPCs get smart enough to realise standing next
to exploding barrels is hazardous, cinematic experience is
diminished.
• Issue 2 : Computational Complexity
7
Issues with GOAP
• Issue 1 : Lack of directorial control.
‣When NPCs get smart enough to realise standing next
to exploding barrels is hazardous, cinematic experience is
diminished.
• Issue 2 : Computational Complexity
‣ GOAP is derived directly from STRIPS. NP-Hard search
problems in the general case.
7
Issues with GOAP
8
Issues with GOAP
• Issue 1 - either a “strong” AI approach is suitable to your
design or it isn’t. Places it often will be include sandbox
environments and companion AI.
8
Issues with GOAP
• Issue 1 - either a “strong” AI approach is suitable to your
design or it isn’t. Places it often will be include sandbox
environments and companion AI.
• Issue 2 is what will be the focus of the rest of the session -
how have planning systems improved since STRIPS/GOAP?
8
Complexity Reduction
9
Complexity Reduction
• If you can reduce complexity of the problem, it
becomes easier to solve...
9
Complexity Reduction
• If you can reduce complexity of the problem, it
becomes easier to solve...
• Either less depth to the problem or less breadth.
9
Landmark Analysis
10
Landmark Analysis
10
Initial State
Landmark Analysis
10
Initial State
Landmark Analysis
10
Initial State
Goal Found
Landmark Analysis
10
Initial State
Goal Found
Landmark Analysis
10
Initial State
Goal Found
Landmark 1
Landmark Analysis
10
Initial State
Goal Found
Landmark 1
Landmark Analysis
10
Initial State
Goal Found
Landmark 1
Landmark 2
Landmark Analysis
10
Initial State
Goal Found
Landmark 1
Landmark 2
Landmark Analysis
10
Initial State
Goal Found
Landmark 1
Landmark 2
Abstraction
11
A
B
E
C D
F
Abstraction
11
A
B
E
C D
F
Abstraction
11
Horizon Management
12
A
B
E
C D
F
Horizon Management
12
Horizon Management
12
E
Horizon Management
12
B
E
Horizon Management
12
B
E
C
Horizon Management
12
B
E
C D
Horizon Management
12
B
E
C D
F
B
E
C D
F
Heuristics
13
Heuristics
• Since GOAP came out, major advances in heuristics
13
Heuristics
• Since GOAP came out, major advances in heuristics
• Most significant :
13
Heuristics
• Since GOAP came out, major advances in heuristics
• Most significant :
‣ Relaxed Plan Graph
13
Heuristics
• Since GOAP came out, major advances in heuristics
• Most significant :
‣ Relaxed Plan Graph
‣ Landmark Heuristic
13
Hierarchical Task Network
14
Hierarchical Task Network
14
Kill Enemy
Hierarchical Task Network
14
Kill Enemy
Approach Enemy Face Enemy Shoot
Enemy
Hierarchical Task Network
14
Kill Enemy
Approach Enemy Face Enemy Shoot
Enemy
Leave Cover Navigate
Hierarchical Task Network
14
Kill Enemy
Approach Enemy Face Enemy Shoot
Enemy
Leave Cover Navigate ...and so on
Hierarchical Task Network
14
Kill Enemy
Approach Enemy Face Enemy Shoot
Enemy
Leave Cover Navigate ...and so on
Until executable actions reached.
Optimality
15
Optimality
•Optimality is a big issue for academic vs industry
15
Optimality
•Optimality is a big issue for academic vs industry
• Academics
15
Optimality
•Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
15
Optimality
•Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
• Industry
15
Optimality
•Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
• Industry
‣ Aim is entertaining - believable, beatable, pseudo-smart
15
Optimality
•Optimality is a big issue for academic vs industry
• Academics
‣ Aim is optimal - shortest, most efficient, least cost
• Industry
‣ Aim is entertaining - believable, beatable, pseudo-smart
• How can we bridge this disconnect?
15
Metrics
16
Metrics
• Plan Metrics allow you to define optimal on your
terms.
16
Metrics
• Plan Metrics allow you to define optimal on your
terms.
• Not a total solution, adds extra compute time.
16
17
But what happens after planning?
17
Plan Execution
18
Plan Execution
• Planning is not the same as doing something
18
Plan Execution
• Planning is not the same as doing something
• Big question is: “what happens next?”
18
Plan Execution
• Planning is not the same as doing something
• Big question is: “what happens next?”
‣ Especially considering that the traditional assumptions of
planning make doing things with plans “challenging”!
18
Execute Blind
19
Execute Blind
19
ExecutePlanStart
PlanEnd
Execute Blind
19
Execute Blind
19
Execute Blind
19
Execute/Replan
20
Execute/Replan
20
ExecutePlanStart
Execute/Replan
20
ExecutePlanStart
??? ?
Execute/Replan
20
ExecutePlanStart
??? ?
Replan
Execute/Replan
20
ExecutePlanStart
??? ?
Execute
Replan
Execute/Replan
20
ExecutePlanStart
??? ?
Execute
Replan
GoalReached
Execution Monitoring
21
Execution Monitoring
21
Integrated Influence
22
Integrated Influence
22
Integrated Influence
22
Integrated Influence
22
Integrated Influence
22
Integrated Influence
22
Integrated Influence
22
Integrated Influence
22
Summary
23
Summary
• GOAP is not the extent of planning
23
Summary
• GOAP is not the extent of planning
•We’ve come a long way in the 40 years since
STRIPS was invented.
23
Summary
• GOAP is not the extent of planning
•We’ve come a long way in the 40 years since
STRIPS was invented.
• Planning is still mostly focused on the “big”
problems.
23
Summary
• GOAP is not the extent of planning
•We’ve come a long way in the 40 years since
STRIPS was invented.
• Planning is still mostly focused on the “big”
problems.
• There is work in planning of relevance.
23
References• Landmarks
‣ “On the Extraction, Ordering and Usage of Landmarks in Planning” Porteous et al, ECP ’01
• Abstraction
‣ “Applying Clustering Techniques to Reduce Complexity in Automated Planning Domains” Dicken &
Levine, IDEAL ’10
• Relaxed Plan Graph
‣ “The FF Planning System: Fast plan Generation Through Heuristic Search” Hoffman, JAIR Vol. 14
• Landmark Heuristic
‣ “The LAMA Planner Using Landmark Counting in Heuristic Search” Richter & Westphal, IPC ’08
• HTNs
‣ “SHOP2 : An HTN Planning System” Nau et al, JAIR Vol. 20
• Execute/Replan
‣ “FF-Replan: A baseline for probabilistic planning” Yoon et al, ICAPS ’07
• Execution Monitoring
‣ “T-REX: A Deliberative System for AUV Control” McGann et al, PPERWS ’07
25