![Page 1: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/1.jpg)
Probabilistic Smart Terrain
Dr. John R. SullinsYoungstown State University
![Page 2: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/2.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
2
Outline
• What is Smart Terrain?
• Why do we need to add probabilities?
• Estimating expected distances to objects that meet character needs
• Plausibility benchmarks and experimental results
• Adding learned knowledge during exploration
• Hierarchical application to games
![Page 3: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/3.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
3
Smart Terrain
• Solves complex navigation problems in real time
![Page 4: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/4.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
4
Smart Terrain
• Characters have “needs”– Example: hunger
• Objects in world meet needs– Example: refrigerator with food inside
• Characters move towards objects that meet needs
![Page 5: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/5.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
5
Smart Terrain
• Objects meets needs transmits “signal”– Signal weakens with distance– Signal moves around objects
![Page 6: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/6.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
6
Smart Terrain
• Characters follow signal to objects– Move in direction of
increasing signal
– Only need to compute map once when level created
– No need for complex navigation!
![Page 7: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/7.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
7
Need for Probabilities
• Smart terrain can result in implausible actions
– Room character has never visited– Contains empty refrigerator
• Does not transmit signal• Character ignores it
– Not plausible behavior!
![Page 8: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/8.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
8
Probabilistic Smart Terrain
• Objects broadcast signal of form“I meet need n”
“I may meet need n with probability P ”
• Probability = uncertainty that object meets need
• Character might explore uncertain objects along path
![Page 9: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/9.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
9
Probabilistic Smart Terrain
• Theoretical goal:Move to closest object with highest probability
• Problem: Optimizing two separate criteria!
• Actual Goal: Plausible behavior for characters
Meets “hunger”
need with P = 0.7
At distance 8
Meets “hunger” need with P = 0.6
At distance 6
![Page 10: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/10.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
10
Expected Distances
• Expected number of tiles character must travel to reach object that fulfills need
• Use to determine which tile to move to next– Compute expected distance for four surrounding tiles– Move to surrounding tile with lowest value for
expected number of tiles to travel
![Page 11: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/11.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
11
Expected Distances
P(t): probability no objects within t tiles meet need
P(t) = (1 – pi ) (Equation 1)
where di < t
• Based on:– di: distances to each object i– pi: probabilities each object i meets need– Assumption of conditional independence
![Page 12: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/12.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
12
Expected Distances
• t < 6: P(t) = 1• 6 ≤ t < 8: P(t) = (1 – 0.6) = 0.4• t ≥ 8: P(t) = (1 – 0.6)(1 – 0.7) = 0.12
Distance: 6 Prob: 0.6
Distance: 8Prob: 0.7
![Page 13: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/13.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
13
Expected Distances
Expected distance from tile T to tile that meets need
E(T) = Σ P(t) (Equation 2) t
t < 6: P(t) = 16 ≤ t < 8: P(t) = 0.4
t ≥ 8: P(t) =0.12
![Page 14: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/14.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
14
Expected Distances
• Problem: Sum could be infinite• Solution: Limit t to some tmax tmax > di i
tmax
E(T) = Σ P(t) (Equation 3)
t
![Page 15: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/15.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
15
Expected Distances
• Compute expected distance E(T) for all tiles T• Character moves to adjacent tile with lowest E(T)
![Page 16: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/16.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
16
Plausibility Benchmarks
• Goal for games:Non-player characters should behave plausibly– Move in direction that “makes sense” to player
• Benchmarks for plausible behavior:– Objects similar in either distance or probability – Group of objects in same direction– Objects that meet need with complete certainty
![Page 17: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/17.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
17
Plausibility Benchmarks• Objects at same distance move to higher probability
• Objects with same probability move to closer one
![Page 18: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/18.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
18
Plausibility Benchmarks• Nearly same distance move to much higher probability
• Nearly same probability move to much closer object
![Page 19: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/19.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
19
Plausibility Benchmarks
Aggregate probabilities benchmark:• Multiple objects > single object with higher probability
– Assumption of conditional independence
![Page 20: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/20.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
20
Plausibility Benchmarks
Complete Certainty benchmark:• Single object with probability = 1 >
multiple objects with probability < 1
![Page 21: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/21.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
21
Learned Knowledge
• Probabilities changed when object reached– Object meets need probability becomes 1– Does not meet need probability becomes 0
• Should affect future actions
Refrigerator empty
Move towards another goal
![Page 22: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/22.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
22
Learned Knowledge• Changing global map affects all characters
– Will also appear to have learned this knowledge
New character enters room
Also ignores empty refrigerator
![Page 23: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/23.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
23
Learned Knowledge
• Each character stores own world model– Belief object meets needs– Initially based on probabilities– Modified when objects explored
Refrigerator R1 70%Refrigerator R2 80%
Object Belief object meets need
0%
![Page 24: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/24.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
24
Learned Knowledge
• Each object propagates raw data to tiles– Probability it meets need– Distance to that tile
![Page 25: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/25.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
25
Learned Knowledge• Character examines surrounding tiles
– Modify probabilities using world model
– Compute expected distances for each
![Page 26: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/26.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
26
Hierarchical Smart Terrain
• More realistic scenario:– Know whether objects meet needs– Don’t know if object is present in given area
• Go to entrance of most likely area• If object present, move to it.• Otherwise, move to another area
![Page 27: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/27.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
27
Hierarchical Smart Terrain
• “Area attractors” at entrances to rooms– Broadcast to entire level– Probability object that meets need is in room– Probability set to 0 when reached by character
• Objects in room– Signal range = size of room– Probability = 1 if present in room
![Page 28: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/28.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
28
Hierarchical Smart Terrain
• Compute expected distances from area attractors• Move to “best” room
![Page 29: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/29.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
29
Hierarchical Smart Terrain
• Object is present in area:– Now in range of object, probability meets need = 1– Character will move directly to object
![Page 30: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/30.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
30
Hierarchical Smart Terrain
• Object is not present in area:– Set probability of area attractor = 0– Character will move to next plausible attractor
![Page 31: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/31.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
31
Conclusions
• Probabilities added to Smart Terrain algorithm
• Characters move to adjacent tile with shortest expected distance to a tile that meets need
• Algorithm produces plausible behavior for benchmarks
• Probabilities overridden by learned knowledge
• Hierarchical algorithm for realistic play
![Page 32: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/32.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
32
Ongoing Work
• Algorithm modification to avoid local minima
• Characters with multiple needs at different levels– Low-probability object that meets critical need– High-probability object that meets less critical need– Which to move towards?
• Objects that change over time– Empty refrigerator now may be restocked in future
![Page 33: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/33.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
33
Local Minima
Caused when paths to low probability objects overlap
Overlap in paths to P=0.23 objects
Tiles nearer to object appear farther away
![Page 34: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/34.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
34
Local Minima
Solution: Weight estimated tiles by distance tmax
E(T) = Σ P(t) k t t
• Nearby objects “appear” even closer
• Any weight k > 0 seemsto work
![Page 35: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/35.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
35
Multiple Needs
• Characters can have multiple needs– Hunger, Fun
• Some needs more critical than others– Hunger = 10 Fun = 5
• Objects may only partially fulfill needs– Donuts: Hunger-7– Cookies: Hunger-3– TV: Fun-6
• Needs increase over time (each tile traversed)– Hunger += 0.5 per tile Fun += 0.2 per tile
![Page 36: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/36.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
36
Multiple Needs
• Goal: Minimize total “discontentment”
Σ (needj)2
j
• Problem: Balancing different factors
![Page 37: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/37.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
37
Multiple Needs• Terminology:
– n j = current level of need j
– di j = distance to object i
– pi j = probability object i meets need j
– ai j = amount that need j decreased by if it meets need)
– c j = increase in need j for each tile traversed
• Expected decrease in need j caused by all objects i = Σ pi
j ai
j within t tiles i
![Page 38: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/38.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
38
Multiple Needs
• Expected level of need j if move t tiles:
max (0, n j + tc j
- Σ pi j ai
j ) where di < t
i
• Expected discontentment if move t tiles:
Σ (max (0, n j + tc j
- Σ pi j ai
j ))2 where di < t j i
![Page 39: Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University](https://reader033.vdocuments.us/reader033/viewer/2022051301/5a4d1b1b7f8b9ab0599938d0/html5/thumbnails/39.jpg)
John Sullins Youngstown State University
Probabilistic Smart Terrain ICTAI 2009
39
Multiple Needs
• Total expected discontentment at given tile:
tmax
Σ Σ (max (0, n j + tc j
- Σ pi j ai
j ))2 t j i
• Compute for surrounding tiles• Move to tile with lowest expected discontentment