terrain runner: control, parameterization, composition ...terrain runner: control, parameterization,...
TRANSCRIPT
![Page 1: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/1.jpg)
Terrain Runner: Control, Parameterization, Composition,
and Planning for Highly Dynamic Motions
Libin Liu* KangKang Yin† Michiel van de Panne‡ Baining Guo§
*Tsinghua University †National University of Singapore
‡University of British Columbia §Microsoft Research Asia
![Page 2: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/2.jpg)
Motivation
[clips are from YouTube, uploaded by 3runTube, l1consolable,
ParkourGenerations, rubenparkour, traceurelements]
2/44
![Page 3: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/3.jpg)
Outline
• Motivation
• Related work
• Controller synthesis pipeline + results
• Conclusion
3/44
![Page 4: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/4.jpg)
Related Work
• Kinematic Methods– [Kovar et al. 2002; Heck and Gleicher 2007;
Min et al. 2009; Treuille et al. 2007; Lee et al. 2009]
• Physics-based Methods– Single controllers: [Hodgins et al. 1995;
Zhao and van de Panne 2005; Muico et al. 2009; Coros et al. 2010; Lee et al. 2010; Wang et al. 2009]
– Control Composition: [Faloutsos et al. 2001, Sok et al. 2007, da Silva et al. 2009, Muicoet al. 2011, Coros et al. 2011]
4/44
![Page 5: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/5.jpg)
System Overview
run
drop roll
jump
vault
single
controller
parameterization
composition
control
online planning
1
2
3
4
parameterized
controllers
single example
motion clips
run
jump
vault
drop roll
5/44
![Page 6: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/6.jpg)
Motion Examples
run
drop roll
jump
vault
single example
motion clips
6/44
![Page 7: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/7.jpg)
composition
control
System Overview
run
drop roll
jump
vault
single example
motion clips
parameterization
run
drop roll
jump
vault
online planning
2
3
4
parameterized
controllers
single
controller
1
7/44
![Page 8: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/8.jpg)
Stage 1: Single Controller Construction
run
drop roll
jump
vault
single example
motion clips
single
controller
1
a: Open loop policy
[Liu et al. 2010]
b: Reduced-order closed-loop
policy [Ding et al. 2012]
8/44
![Page 9: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/9.jpg)
Stage 1a: Open-loop Policy
[Liu et al. 2010]: Sampling-based Contact-rich Motion
Control, SIGGRAPH 2010
𝑎
𝑠
reference states
reference actions
9/44
![Page 10: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/10.jpg)
Stage 1b: Reduced-order Closed-loop Policy
𝑠 − 𝑠 = 𝛿𝑠
𝛿𝑎 = 𝑎 − 𝑎
[Ding et al. 2012]: Learning reduced-order feedback policies
for motion skills. Tech. Rep. University of British Columbia.
𝛿𝑎 = 𝑀 𝛿𝑠 + 𝑎
10/44
![Page 11: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/11.jpg)
Stage 1b: Reduced-order Closed-loop Policy
𝑠 − 𝑠 = 𝛿𝑠
𝛿𝑎 = 𝑀 𝛿𝑠 + 𝑎
change in control change in states
𝑀𝑎𝑝 ⋅ 𝑀𝑠𝑝
𝛿𝑎 = 𝑎 − 𝑎
11/44
![Page 12: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/12.jpg)
Stage 1b: Reduced-order Closed-loop Policy
𝛿𝑠
𝛿𝑎 = 𝑀 𝛿𝑠 + 𝑎
change in control change in states
𝑀𝑎𝑝 ⋅ 𝑀𝑠𝑝
𝛿𝑎
𝑀𝑠𝑝𝑀𝑎𝑝
reduced-order state
12/44
![Page 13: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/13.jpg)
Stage 1b: Feedback Policy
Manually-selected States: s
• Running: 12 dimensions
{ , , , }
13/44
![Page 14: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/14.jpg)
Stage 1b: Feedback Policy
Manually-selected Controls: a
• for all skills: 9 dimensions
14/44
![Page 15: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/15.jpg)
• Vaulting
phase 1: raising phase 2: dropping phase 3: landing
Stage 1b: Feedback Policy
Multi-phase Skills
15/44
![Page 16: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/16.jpg)
Stage 1b: Feedback Policy
Multi-phase Skills
• Drop-rolling
phase 1: jumping phase 2: dropping phase 3: rolling phase 4: standing-up
16/44
![Page 17: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/17.jpg)
• Jumping
• Vaulting
• Drop-rolling
Stage 1b: Feedback Policy
Manually-selected States: s
}{
{ }
}{
17/44
![Page 18: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/18.jpg)
• Optimize 𝑀
– CMA, Covariance Matrix Adaption ([Hansen 2006])
– Running:
• Objective function
• 12 minutes on 24 cores
• more details in paper and [Ding et al. 2012]
𝛿𝑎 = 𝑀𝛿𝑠 + 𝑎
𝐸 = 𝑤𝑡 𝑁𝑑𝑇𝑐 − 𝑇𝑠 + 𝑤𝑠𝐸𝑠 + 𝑤𝑝𝐸𝑝 + 𝑤𝜏𝐸𝜏
Stage 1b: Feedback Policy
Optimization
18/44
![Page 19: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/19.jpg)
composition
control
single
controller
1
System Overview
run
drop roll
jump
vault
single example
motion clips
online planning
3
4
parameterization
2
run
drop roll
jump
vault
parameterized
controllers
19/44
![Page 20: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/20.jpg)
𝛿𝑎 = 𝑀 𝛿𝑠 + 𝑎
Stage 2: Parameterization
𝛿𝑎 = 𝑀𝜃 𝛿𝑠 + 𝑎𝜃
𝑎𝜃
𝜃
20/44
![Page 21: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/21.jpg)
Stage 2: Parameterization
Running: parameter space
• 𝜃 = (v, 𝜙)
– speed, turning rate
– [2m/s, 5m/s] × [−6°, 6°]/step
21/44
![Page 22: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/22.jpg)
𝒓𝒍
𝑡 𝑡 𝑡 𝑡
Stage 2: Parameterization
Running: Action Set Augmentation
• 𝑎𝜃 = , , 𝛼 , 𝛽
space scaling time scaling
22/44
![Page 23: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/23.jpg)
Stage 2: Parameterization
Running: optimization
• 𝑀𝜃 , 𝑎𝜃
success head’s stability
desired parameters
23/44
![Page 24: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/24.jpg)
Stage 2: Parameterization
Continuation
• [Yin et al. 2008]: Continuation methods for adapting
simulated skills. SIGGRAPH 2008Radial Basis Functions
24/44
![Page 25: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/25.jpg)
Stage 2: Parameterization
Continuation
• Predictor-corrector
…
25/44
![Page 26: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/26.jpg)
Stage 2: Parameterization
Running Results
26/44
![Page 27: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/27.jpg)
Stage 2: Parameterization
Obstacle Clearing Maneuvers
• 𝜃 = ℎ
– Obstacle height
• Jumping – [0.1m, 0.7m]
• Vaulting – [0.6m, 1.0m]
• Drop-rolling – [0.9m, 2.0m]
27/44
![Page 28: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/28.jpg)
Stage 2: Parameterization
Obstacle Clearing Maneuvers
• Optimization
– 𝑎ℎ = { , 𝛼, 𝛽}
contact balance pose
28/44
![Page 29: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/29.jpg)
Stage 2: Parameterization
Obstacle Clearing Maneuvers
• Jumping
– Contact term 𝐸𝑐
ℎ ℎ𝑓
29/44
![Page 30: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/30.jpg)
ℎ𝑟 ℎ𝑑ℎ ℎℎ
𝑑ℎ
Stage 2: Parameterization
Obstacle Clearing Maneuvers
• Vaulting
– Contact term 𝐸𝑐
30/44
![Page 31: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/31.jpg)
Stage 2: Parameterization
Obstacle Clearing Maneuvers
• Drop-rolling
– Balance term 𝐸𝑏
𝑑𝑟 𝑑𝑙
𝑑
𝐿
31/44
![Page 32: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/32.jpg)
Stage 2: Parameterization
Obstacle Clearing Results
32/44
![Page 33: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/33.jpg)
single
controller
1
System Overview
run
drop roll
jump
vault
single example
motion clips
parameterization
online planning
2
4
parameterized
controllers
run
drop roll
jump
vault
3
composition
control
33/44
![Page 34: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/34.jpg)
Stage 3: Composition
Three-phase Composition Scheme
clear
transIn transOutrun run
Three-phase Composition Scheme
𝛿𝑎 = 𝑀𝜃𝑖𝑛
′ 𝛿𝑠 + 𝑎𝜃𝑖𝑛𝛿𝑎 = 𝑆𝜃𝑜𝑢𝑡
𝛿𝑠
𝛿𝑎 = 𝑀ℎ′ 𝛿𝑠 + 𝑎ℎ
Obstacle
𝛿𝑎 = 𝑀𝜃𝑖𝑛𝛿𝑠 + 𝑎𝜃𝑖𝑛
𝛿𝑎 = 𝑀𝜃𝑜𝑢𝑡𝛿𝑠 + 𝑎𝜃𝑜𝑢𝑡
clear
34/44
![Page 35: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/35.jpg)
Stage 3: Composition
Three-phase Composition Scheme
35/44
![Page 36: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/36.jpg)
Stage 3: Composition
Optimization
• Parameters
– 𝜃𝑖𝑛, 𝑀𝜃𝑖𝑛
′ , 𝑀ℎ′ , 𝑆𝜃𝑜𝑢𝑡
transIn transOut
𝛿𝑎 = 𝑀𝜃𝑖𝑛
′ 𝛿𝑠 + 𝑎𝜃𝑖𝑛𝛿𝑎 = 𝑆𝜃𝑜𝑢𝑡
𝛿𝑠
𝛿𝑎 = 𝑀ℎ′ 𝛿𝑠 + 𝑎ℎ
run run
clear
Three-phase Composition Scheme
36/44
![Page 37: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/37.jpg)
Stage 3: Composition
Results
37/44
![Page 38: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/38.jpg)
single
controller
1
System Overview
run
drop roll
jump
vault
single example
motion clips
parameterization
run
drop roll
jump
vault
composition
control
online planning
2
3
4
parameterized
controllers
38/44
![Page 39: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/39.jpg)
Stage 4: Online Planning
• Step-based kinematic planning
min𝑤𝑢 𝑢 − 𝑢ℎ2 + 𝑤𝑖 𝑣𝑖 − 𝑣𝑖−1
2
s. t. 𝑣𝑛 = 𝑣𝑖𝑛
𝑢, 𝑣𝑛
𝑣𝑖𝑛
𝑢ℎ
𝑢, 𝑣𝑛
39/44
![Page 40: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/40.jpg)
Stage 4: Planning
Results
40/44
![Page 41: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/41.jpg)
Results
Terrain Running
41/44
![Page 42: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/42.jpg)
Conclusion
• Parkour-style motions
– Running, jumping, vaulting, drop-rolling
• Complete framework, Realtime synthesis
– Control construction, parameterization, composition,
planning
• Structured optimization scheme
42/44
![Page 43: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/43.jpg)
Limitations
• Only partly automated
• Composition can fail
• No arbitrary transitions
43/44
![Page 44: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/44.jpg)
Future Work
[Parkour Memories, uploaded by 3runTube
http://www.youtube.com/watch?v=24cgnAA6x0I&hd=1]
44/44
![Page 45: Terrain Runner: Control, Parameterization, Composition ...Terrain Runner: Control, Parameterization, Composition, and Planning for Highly Dynamic Motions Libin Liu* KangKang Yin†](https://reader034.vdocuments.us/reader034/viewer/2022052310/606fb1530f494203aa620e76/html5/thumbnails/45.jpg)
Thanks