![Page 1: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/1.jpg)
Motion Synthesis for Articulated Human Bodies
![Page 2: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/2.jpg)
Contents
• Human motion synthesis: Zoo
• Articulated body mechanicsArticulated body mechanics
• Optimization based motion synthesisOptimization based motion synthesis
• Feedback based balance controlFeedback based balance control
![Page 3: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/3.jpg)
Human motion components
• Bone (skeleton): ~206 (motion invariant)• Joint (articulation): limitation• Tendon and ligament: elasticity / plasticity• Muscle: ~639, works in group, limitation
– uni-articulate and bi-articulate (passive inefficiency)
– parallel or perpendicular shape
• Neuron-system: reflection speedNeuron Muscle Tedon Bone
![Page 4: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/4.jpg)
![Page 5: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/5.jpg)
Human motion model (I)
• Pure articulated model– Simplified skeleton (hand / foot / tibia-fibula)– Directly control force and torque on joints– Convenient for high-level control
![Page 6: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/6.jpg)
Human motion model (II)
• Deformable model– Skin mesh motored by skeleton or captured
skin animation– Muscle shape representation
![Page 7: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/7.jpg)
Human motion model (III)
• Tendons and muscles model– Musculoskeletal models and simulation– Muscle-tendon model (strand)
Hand can have similar DoF with coarse whole body!
![Page 8: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/8.jpg)
Human motion model (IV)
• Visible Human data set (realistic musculature and flesh) [Teran05]
![Page 9: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/9.jpg)
Methods for motion synthesis (I)
• Data-driven based synthesis– Captured motion skeleton data– Captured deformable data (face / skin)– Motion dynamics and naturalness is trivial– Limited response to environment
• Blending, Editing, Style transfer; no guarantee for physics correctness
– Captured data is sparse• [pose, time] space is huge
– Difficult to obtain data• High dynamic motion• Outdoor environment
![Page 10: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/10.jpg)
Methods for motion synthesis (II)
• Pure generative-based synthesis– Physics constraint can be guaranteed – Easy to interact with dynamic environment– Hard to be natural
• Naturalness is a subset of physics correctness
– Dynamics simulator complexity• E.g. stable / dynamic frictions
– Computational complexity and stableness• High gain means high stiffness
– Controller design • Mutual controller is difficult to design and hard to generalize
![Page 11: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/11.jpg)
Methods for motion synthesis (III)
• Pure motion planning– Can obtain solution in constrained
environment– High-level path planning– Solution has no continuous or physics
guarantee – Still time consuming
• High dimension of human skeleton
![Page 12: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/12.jpg)
Combined methods
• Mocap data + physics: better controller– Controller for hand [Nancy05]
– Controller for whole body [Yin08]
• Mocap data + motion planning: high level motion: – Manipulating [Katsu04]
– Tangling [Edmond08]
• Dynamics + motion planning (for articulated model) [Russell07]
![Page 13: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/13.jpg)
Other related
• Robotics [Kris06]
– Primitive based motion planning use transition motion to balance natural motion and environment constraints.
• Biomechanics– Provides many principles
for motion control: balance, Locomotion, neuroscience. [Alexandrov05] [Christine07]
![Page 14: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/14.jpg)
Contents
• Human motion synthesis: ZooHuman motion synthesis: Zoo
• Articulated body mechanics
• Optimization based motion synthesisOptimization based motion synthesis
• Feedback based balance controlFeedback based balance control
![Page 15: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/15.jpg)
Articulated body dynamics
• Ball-and-socket: 3 DoF• Saddle: 2 DoF• Hinge: 1 Dof
• Each joint i is 1 DoF• Ball-and-socket and saddle can
be represented by some 1 DoF
![Page 16: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/16.jpg)
A special case of multi-body dynamics
Yin’s work gives
an example
Joint makes impulse
and penalty difficult
![Page 17: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/17.jpg)
Parameters for articulated body
• State of root (position, rotation, linear / angular velocity)
• Configuration of joints (q, , )
• Inertia matrix (for articulated body, inertia matrix is an equivalent inertia matrix: give a test force, get an acceleration inertia)
• Lecture in COMP 790-058
![Page 18: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/18.jpg)
Inverse dynamics
• Known: q, ,
• Unknown: f, τ• Simple: recursive [Featherstone]
– Compute v and a; compute net force on a link, similar to f = ma (downwards)
– Compute force and torque on a joint (upwards)
– Root’s p, v, a – End effector’s force and torque
![Page 19: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/19.jpg)
Forward dynamics
• Known: q, , f, τ
• Unknown:
• More difficult– 3 loops
[Evangelos04]
![Page 20: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/20.jpg)
Limitation of basic algorithms
• Many things are simplified in algorithms– Joint limitation– Unilateral constraint (ground constraint)– Friction (kinetic, static, rolling and spinning)– Non-interpenetrate constraint– Collision response
• Modeling contact and constraint is a well-studied problem for rigid body simulation. – See David Baraff’s papers and note.
![Page 21: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/21.jpg)
Linear complementarity problem (LCP) [Andreas]
• One of the standard methods to handle contact and constraints.
![Page 22: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/22.jpg)
Solution to LCP: pivot method
• Basic idea: if q is positive or zero, solution is trivial: (w = q, z = 0)
• Pivot q and w to make it true
• where
![Page 23: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/23.jpg)
Solution to LCP: iterative method
• LCP can be represented by QP, so can be solved by iterative methods, like Gauss-Seidel, Newton
![Page 24: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/24.jpg)
Pros and Cons
• Pivot method– Convergence is guaranteed after limited steps– suffer from numerical problem, especially for
large-scale and/or ill-conditioned problems
• Iterative method– easier to implement and– numerically robust– convergence is proven only for a limited class
of M matrix
![Page 25: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/25.jpg)
LCP model for constraints
• Unilateral constraint (force exists only when contacting)
• Joint Limit
• LCP guarantees zero virtual work for contact forces
• w = Mz + q?
![Page 26: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/26.jpg)
Extended LCP model for frictions
• Coulomb cone
• Dynamic friction Static friction
• Solved by extending iterative LCP solver
![Page 27: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/27.jpg)
Other possible solutions
• Recent there are many new algorithms other than LCP to deal with unilateral constraints and friction– Nonconvex Rigid Bodies with Stacking [Eran03]
• Extension to LCP– Staggered Projections for Frictional Contact in
Multibody Systems [Danny08]
– Velocity based shock propagation[Kenny09]
– Implicit Contact Handling for Deformable Objects [Miguel09]
![Page 28: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/28.jpg)
Impact constraints
• Non-interpenetrate constraint
• Collision reaction
• Impulse or force modeling
• See David Baraff’s papers and note.
![Page 29: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/29.jpg)
Simulation loops
LCP solver
forward dynamics
Ode solver
Collision detector
Contact response
![Page 30: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/30.jpg)
Contents
• Human motion synthesis: ZooHuman motion synthesis: Zoo
• Articulated body mechanicsArticulated body mechanics
• Optimization based motion synthesis
• Feedback based balance controlFeedback based balance control
![Page 31: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/31.jpg)
Optimization based motion synthesis [Sumit09]
![Page 32: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/32.jpg)
System overview
![Page 33: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/33.jpg)
Lagrange mechanics
• Lagrange mechanics for rigid body
• Lagrange mechanics for articulated body
Lagrange function
Generalized force
Gravity, contact force, external force
![Page 34: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/34.jpg)
Complete force modeling [Liu05]
![Page 35: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/35.jpg)
Actuated and non actuated joint
• Root joint: non (passive) actuated– It’s configuration (position and rotation) is
actuated by joint constraints.
• Other joints: (active) actuated– It’s configuration is actuated by muscle
energy
Muscle force
![Page 36: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/36.jpg)
Passive controller
• Muscle controller– Limited torque / force
– Limited torque / force change rate
• Contact controller– Static friction– Dynamic friction– Non-penetration
![Page 37: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/37.jpg)
Optimization with passive controller
• Why (minimize q variation)?– Avoid trivial solution, like sliding or break-off
• Zero virtual work guarantee– Roll back
![Page 38: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/38.jpg)
User controller specification
• Balance controller
• Climb controller
• Swing controller
• Multiple controller composition
• controller example
![Page 39: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/39.jpg)
Controller protocol
• Finite State Machine
• Each state has its own constraints (in objective form)
• State transition happens when feasible solution can not find or contact breaks.
![Page 40: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/40.jpg)
Balance controller
balance
takeStep relaxFoot
p: target position
qv: slow
spline: perpendicular to ground
cp: CoM balance
cf: support feet position
c: friction
![Page 41: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/41.jpg)
Climb controller
allSupport
moveHand
relaxHand
moveFoot
relaxFoot
com: change CoM
![Page 42: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/42.jpg)
Swing controller
passiveSwingtrySwing
![Page 43: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/43.jpg)
Visual sensor
• Reachable objects search
![Page 44: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/44.jpg)
Contents
• Human motion synthesis: ZooHuman motion synthesis: Zoo
• Articulated body mechanicsArticulated body mechanics
• Optimization based motion synthesisOptimization based motion synthesis
• Feedback based balance control
![Page 45: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/45.jpg)
Human motion control model
![Page 46: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/46.jpg)
Feedback based balance control
• Balance control under small perturbation [Yin03]
• Balance control under large perturbation [Yin08]
• Similar strategy: feed-back + feed-forward, large perturbation step strategy
![Page 47: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/47.jpg)
System description
• The dynamics system does not use generalized coordinate
• Instead, use full-coordinate constrained form for dynamics
• Traditional style (based on Barraf 1996 paper)
• Control is a hybrid of generalized-coordinate form and full-coordinate form
![Page 48: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/48.jpg)
Full-coordinate constrained matrix form (I)
• Constraint i between body a and b
• Constraint system (each item in J is 3*3 matrix)
Here f is dim-12 vector, every 3
sub-vector is force for body (a, b, c, or d), J is 9*12
![Page 49: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/49.jpg)
Full-coordinate constrained matrix form (II)
• For torque, similar
• row number of H is number of freedom, its column number is 3 * object number.
• Each term is joint i’s torque on all objects
![Page 50: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/50.jpg)
Balance control under small perturbation
• Preprocess: use inverse dynamics to compute force for original motion
• In each step of dynamic simulation– Forward dynamics: current state– Feedback control
– Feedforward control
– Net control
![Page 51: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/51.jpg)
Simulation result
• Perturbation by a ball
![Page 52: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/52.jpg)
Balance control under large perturbation
• Two differences:– Motion is modeled by finite state machine– Under large perturbation, a pre-computed
feed-forward input is not suitable (using adaptive control instead)
![Page 53: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/53.jpg)
Motion FSM
• Difference with FSM in [Sumit09]
– For control torque heuristics– More robust (precision, static)
![Page 54: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/54.jpg)
Feed-forward: Feed-back error learning
• Cyclic motion (function of phase)
• Learn inverse model dynamically (system identification)
![Page 55: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/55.jpg)
Feed-back control (I)
• PD control
• Contact control (support leg)
Low gain
Follow mocap data: high gain
![Page 56: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/56.jpg)
Feed-back control (II)
• COM feedback
+v
+v
+
d<0
v=0+
d>0
v=0
COM velocity
COM position
v>0 or d > 0 forward step quickly
![Page 57: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/57.jpg)
Feed-back control (III)
• CoM feedback
Basic controller(default target)
Continuous feedback
General form for multiple joints (matrix form)
![Page 58: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/58.jpg)
Control summary
• 2D to 3D:– sagittal and coronal planes
Feedback and feedforward torques
![Page 59: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/59.jpg)
Simulation result
• demo• Overview• Downhill• Drunk• Limp• Spin• Boxes• Different friction
![Page 60: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/60.jpg)
Reference• [Edmond08] Planning tangling motions for humanoids• [Nancy05] Physically Based Grasping Control from Example• [Yin08] SIMBICON: Simple Biped Locomotion Control• [Katsu04] Synthesizing Animations of Human Manipulation Tasks• [Russell07] Efficient Motion Planning of Highly Articulated Chains using Physics-
based sampling• [Kris06] Using motion primitives in probabilistic sample-based planning for humanoid
robots• [Teran05] Creating and Simulating Skeletal Muscle from the Visible Human Data Set• [Alexandrov05] Feedback equilibrium control during human standing• [Christine07] Bipedal locomotion: toward unified concepts in robotics and
neuroscience• [Evangelos04] Practical Physics for Articulated Characters• [Shin’schiro07] Constraint-based dynamics simulator for humanoid robots with shock
absorbing mechanics• [Featherstone] Robot Dynamics Algorithms• [Andreas] Practical Optimization• [Eran03] Nonconvex Rigid Bodies with Stacking• [Danny08] Staggered Projections for Frictional Contact in Multibody Systems
![Page 61: Motion Synthesis for Articulated Human Bodies](https://reader035.vdocuments.us/reader035/viewer/2022081513/56813aff550346895da39816/html5/thumbnails/61.jpg)
• [Sumit09] Optimization-based interactive motion synthesis• [Liu05] Towards a generative model of natural motion• [Baraff89] Analytical methods for dynamics simulation of non-penetrating rigid bodies• [Yin03] Motion Perturbation Based on Simple Neuromotor Control Models• [Kenny09] Velocity-based shock progagation for multibody dynamics animation• [Miguel09] Implicit Contact Handling for Deformable Objects