SIGGRAPH Asia 2011

Preview Seminar

Session 11: Animation2011/11/25

Jun SaitoMarza Animation Planet, Inc.

Session 11: AnimationFacial Animation

Artist Friendly Facial Animation Retargeting

Compression and Direct Manipulation of Complex Blendshape Models

Simulated Character

Controlling Physics-Based Characters Using Soft Contacts

Modal-Space Control for Articulated Characters

Skinning Stretchable and Twistable Bones for Skeletal Shape Deformation

Artist Friendly Facial Animation Retargeting

Yeongho Seol∗, Jaewoo Seo†, Paul Hyunjin Kim‡, J.P. Lewis§, Junyong Noh¶

∗†‡¶KAIST, §Weta Digital

Artist Friendly Facial Animation Retargeting

MotivationRetargeting from facial motion capture does not yield enough efficiency

Method Manual Key-framing Retargeting Edits

Total time 40 min. 36 min.

Time required to edit 135 frames of animation

Side Note:Facial Retargeting EfficiencyTechnology X used at Marza

Method Manual Key-framing Retargeting Edits

Total time 24 hrs. 5 hrs.

Artist Friendly Facial Animation Retargeting

System Overview

Automatic GUI

Generation

Sequential Retargeting

Graph Simplificati

on

Artist Friendly Facial Animation Retargeting

Automatic GUI Generation

• Facial rig w/ arrow-like GUIo Compute dominating motion vector of each blendshape target by

WPCA

• Grouping of GUI to regions and layerso Manually paint four regions: mouth, forehead, eyes, and otherso Three layers (large, mid, small) depending on the total

displacement from neutral face

Artist Friendly Facial Animation Retargeting

RetargetingCompute marker correspondence (generic face to actor’s face)by RBF warping

Artist Friendly Facial Animation Retargeting

RetargetingConventional (non-sequential) retargeting

Capturedmarker positions

Marker positionsof blendshapes

Artist Friendly Facial Animation Retargeting

Sequential retargeting

• Solve NNLS for blendshape target starting with largest total displacement

• Mimics animators’ workflow (coarse posing, then fine tuning)

Retargeting

Artist Friendly Facial Animation Retargeting

1. Find optimal salient points using dynamic programming

2. Find piecewise optimal Bezier curve

Graph Simplification

Artist Friendly Facial Animation Retargeting

Results　 GUI Seq. Graph Test 1 Test 2 Avg. Time

Face 1 　 　 　 　 　(a) Manual key-framing 　 36 42 39

(b) Y N N 32 36 34

(c) Y N Y 30 36 33

(d) Y Y N 18 20 19

(e) Y Y Y 15 19 17

Face 2 　 　 　 　 　(a) Manual key-framing 　 26 23 24

(b) Y N N 18 18 18

(c) Y N Y 17 15 16

(d) Y Y N 9 7 8

(e) Y Y Y 8 5 6

Minutes required to edit 135 frames of animation

Compression and Direct Manipulation of Complex

Blendshape ModelsJaewoo Seo∗, Geoffrey Irving†, J.P. Lewis‡, Junyong Noh§

∗§KAIST, †‡Weta Digital

Compression and Direct Manipulation of Complex Blendshape Models

Facial Blendshapes @ Weta:o 42,000 verticeso 730 targetso Density close to 100%o 8 fps on 8 core CPU

Generic model for feature-quality, semi-realistic character• 6,821 vertices• 96 targets• Average density: 10% (max 30%)

Perhaps sparse matrix implementation is enough?

Side Note:Blendshapes @ Marza

Compression and Direct Manipulation of Complex Blendshape Models

Make humongous blendshapes more tractablewith lossy matrix compression using

hierarchical semi-separable (HSS) representation

Technique can be applied to compression and speed-up of large matrix multiplication

Contributions

Compression and Direct Manipulation of Complex Blendshape Models

CompressionPreparation: reordering

o Place high-rank blocks on “diagonal,” low-rank blocks on “off-diagonal”o Exact permutation is NP hardo Find bisection by minimizing crossing weight below, use heuristic in

[Kernighan and Lin 1970]

Compression and Direct Manipulation of Complex Blendshape Models

CompressionHierarchical Semi-Separable (HSS) Representation

Tree of orthogonal matrices to compresssignificant off-diagonal structure

Compression and Direct Manipulation of Complex Blendshape Models

CompressionHSS Construction [Xia et al. 2010]

Compression and Direct Manipulation of Complex Blendshape Models

CompressionHSS Construction [Xia et al. 2010]

Compression and Direct Manipulation of Complex Blendshape Models

CompressionHSS Construction [Xia et al. 2010]

Compression and Direct Manipulation of Complex Blendshape Models

CompressionHSS Construction [Xia et al. 2010]

Compression and Direct Manipulation of Complex Blendshape Models

CompressionHSS Construction [Xia et al. 2010]

Compression and Direct Manipulation of Complex Blendshape Models

Compression• Perform SVD on matrix blocks, drop singular

values

• Optional: represent rotation using banded Householder factorization [Irving 2011]

Compression and Direct Manipulation of Complex Blendshape Models

Parallel ProcessingGPU-optimized HSS-compressed matrix multiplication

Compression and Direct Manipulation of Complex Blendshape Models

More Applications• Direct manipulation

[Lewis and Anjyo 2010] with local influence

• Cage deformation

Compression and Direct Manipulation of Complex Blendshape Models

　 Matrix Size Dense Sparse PCA Local PCA HSS HSS+Banded

Character # Rows # Cols MB % MB % MB % MB % MB % MB %

Dumb 127173 730 354 100% 348 98.3% 138.7 39.2% 87.4 24.7% 46.8 13.2% 25.4 7.2%

Dumber 155187 625 370 100% 317 85.7% 164.6 44.5% 104.7 28.3% 46 12.4% 28.1 7.6%

Armadillo 106289 284 115.2 100% 173.2 150.3% 114.8 99.7%　 　 10.4 9.0% 8.6 7.5%

τ=1e-2 for PCA, τ=1e-3 for HSS

ResultsMemory

Speed (in milliseconds)　 Sparse HSS HSS+Banded

Character 8 CPUs 8 CPUs GPU 8 CPUs GPU

Dumb 124.21 11.22 1.47 10.73 2.18

Dumber 113.65 11.01 1.41 10.76 2.12

Armadillo 28.78 3.62 0.82 4.22 1.18

Controlling Physics-Based Characters Using

Soft ContactsSumit Jain and C. Karen Liu

Georgia Institute of Technology

Controlling Physics-Based Characters Using Soft Contacts

Motivation

VIDEO

Controlling Physics-Based Characters Using Soft Contacts

Contributions• Coupling of articulated rigid body and soft

body with practical contact model

• Experiments to show soft contact model stabilizes physically simulated characters

Controlling Physics-Based Characters Using Soft Contacts

Coupled Dynamics• Articulated rigid body

Massmatrix

Coriolismatrix

Gravity Generalizedforces

Jacobianat contact

Contactforce

Controlling Physics-Based Characters Using Soft Contacts

• Deformable bodyo Vertex deformation – tries to keep vertices at their rest positions

o Edge deformation – tries to keep the relative positions of the vertices

Coupled Dynamics

Stiffnessmatrix

Dampingmatrix

Controlling Physics-Based Characters Using Soft Contacts

Coupled Dynamics• Rigid + Deformable

• Adaptive deformable modelo P-ring neighborhood of contact points are

simulated, rest are treated as rigido Mass matrix is pre-computed at the rest positions

Controlling Physics-Based Characters Using Soft Contacts

Coupled Dynamics• Discretize with time step h

Controlling Physics-Based Characters Using Soft Contacts

Contact Model• Contact with friction as Linear Complementarity

Problem (LCP) [Anitescu and Potra 1997]o Advantages over penalty-based methods:

• Enforces work-less normal force, no penetration, and realistic slipping

• Explicit deformation at contact increases contact points• Low stiffness in penalty methods causes frequent penetration

Controlling Physics-Based Characters Using Soft Contacts

Locomotion Control:SIMBICON

1. Compute joint torque τs

2. Detect collisions3. Create contacts to be solved for in ODE4. Apply τs to character in ODE

5. Advance one time step in ODE to get next state

Controlling Physics-Based Characters Using Soft Contacts

Locomotion Control:SIMBICON + Proposed Method

1. Compute joint torque τs

2. Detect collisions3. Create contacts to be solved for in ODE

a. Convert τs to generalized torques τr

b. Convert state to generalized coordinatesc. Solve (qk+1, qk+1) and fc using LCP

d. Apply fc to character in ODE

4. Apply τs to character in ODE

5. Advance one time step in ODE to get next state

Modal-Space Control for Articulated

CharactersSumit Jain and C. Karen Liu

Georgia Institute of Technology

Modal-Space Control for Articulated Characters

MotivationArbitrary character simulation with both• long-term (anticipatory) planning• short-term (reactive) planningis prohibitively expensive to compute

?External

force

Modal-Space Control for Articulated Characters

ContributionFormulation of modal-space character control capable of long-term planning and frequent re-planning for simulating“specific motion sequence”

Modal analysis provides:• Independent control: N-dimensional optimization to N

independent one dimensional problems• Model reduction: Allows construction of a small number of

strategies specific to certain frequencies of the dynamic system

Modal-Space Control for Articulated Characters

Modal Analysis

Md and Kd are diagonal, thus

N independent 1d problems

Modal-Space Control for Articulated Characters

Strategy For Modes1. Rigid modes (zero frequency)

o Corresponds to six eigenvectors in Φ with zero eigenvalueso Only affected by external forceso For long term planning

2. Low frequency modeso Corresponds to eigenvectors with

eigenvalues smaller than some thresholdo Visibly significant movements

by actuated system(see right)o For long term planning

3. High frequency modeso Less visually significanto For reactive planning

1st three principal componentsdominated by low frequency

Modal-Space Control for Articulated Characters

Control Summary2. Low frequency

modes: estimate joint actuationo Solve unconstrained QP for

ideal actuation Ia* to match low frequency reference state

1. Rigid body modes: estimate contact forceso Solve QP, constrained by

Coulomb friction, for ideal contact forces f* to match rigid mode reference state

Modal-Space Control for Articulated Characters

Control Summary4. Corrective forces

o Δf and ΔIa are computed to satisfy Coulomb friction

3. High frequency modes: track short-horizon plano Analytically compute high-

frequency actuation to match reference state

Modal-Space Control for Articulated Characters

Results

VIDEO

Stretchable and Twistable Bones for Skeletal Shape

DeformationAlec Jacobson∗ and Olga Sorkine†

∗New York University, †ETH Zurich

Stretchable and Twistable Bones for Skeletal Shape Deformation

Motivation• Conventional bone skinning methods cause

unnatural deformation at the end of bones when scaled

Stretchable and Twistable Bones for Skeletal Shape Deformation

Linear Blend Skinning (LBS)

• Weight for each bone

Resultingposition

Weight foreach bone

Rotation

Scale Originalposition

Stretchable and Twistable Bones for Skeletal Shape Deformation

Stretchable, Twistable Bones Skinning

(STBS) • Weight for each bone and endpoint

o Naturally extends to use dual quaternion

Twist alongbone

Weight foreach endpoint

Stretchable and Twistable Bones for Skeletal Shape Deformation

Results

Stretchable and Twistable Bones for Skeletal Shape Deformation

• Manually paint• Automatic

o Bone Heat [Baran and Popovic 2007]o Bounded Biharmonic Weight [Jacobson et al. 2011]

Generating Weights

Stretchable and Twistable Bones for Skeletal Shape Deformation

Comparison:BH vs. BBW