Real-Time Enveloping with Rotational Regression

Real-Time Enveloping with Rotational Regression

Robert WangKari PulliJovan Popović

Robert WangKari PulliJovan Popović

Enveloped (skinned) characters are pervasive.Enveloped (skinned) characters are pervasive.

Skeletons are often used to control meshes.

skeleton mesh

Physically based modeling provides realistic deformations. Physically based modeling provides realistic deformations. Realistic deformations

– Finite-element based [Teran et al. 2005]

– Anatomy based [Scheepers et al. 1997]

– Elastically deformable [Capell et al. 2002, 2005]

– Used in movie production

– Off-the-shelf commercial tools

Slow evaluation

[Teran et al. 2005]

[Absolute Character Tools 1.6]

We learn a fast model from exported examples.We learn a fast model from exported examples.

Exported Examples

(skeleton-mesh pairs)

Fast Model

Our method

Black Box Simulation

Artists can still use existing modeling tools or scanned data.Artists can still use existing modeling tools or scanned data.

Exported Examples

(skeleton-mesh pairs)

Fast Model3-D Scan Data

This is analogous to mesh simplification.This is analogous to mesh simplification.

High-resolution mesh

Low-resolution mesh

mesh simplification

Higher quality

Used in movie production

Faster to render

Optimized for interactive applications

Physical simulation Rotational Regression Enveloping

learning

How do we map a skeleton to a mesh?How do we map a skeleton to a mesh?

?

What parameters should we learn?

How to model muscle deformations for fast evaluation?

Linear blend skinning linearly maps joint rotations to vertex positions.Linear blend skinning linearly maps joint rotations to vertex positions.

Most popular enveloping technique for games

Coarse modeling parameters (but very simple)

Not very expressive (but very fast)

+

y

y

Figure from [Wang and Phillips 2002]

Linear blend skinning has many names.Linear blend skinning has many names.

Also known as,

– Single-Weight Enveloping

– Skeletal Subspace Deformation (SSD)

– Or just, “Skinning”

We will use “Linear Blend Skinning” or “SSD.”

The two steps of our work are deformation gradients prediction and mesh reconstruction.The two steps of our work are deformation gradients prediction and mesh reconstruction.

Mesh reconstruction

Deformation gradients prediction

(Rotational Regression)

We present a replacement for linear blend skinning.We present a replacement for linear blend skinning.

Coarse modeling parameters.

Can’t handle certain types of deformations.

Fast

Lets you use your existing modeling tool.

Good for muscle bulges.

Fast

Whenever you have an existing model, you Whenever you have an existing model, you should use our technique instead of linear blend should use our technique instead of linear blend skinning.skinning.

+

Rigid components move with the bone rotation

Other surfaces rotate in the opposite direction

Our model is inspired by the behavior of a flexing bicep.Our model is inspired by the behavior of a flexing bicep.

Surface rotation

Bone rotation

Angle is scaled by u. Axis is offset by rotation W.Angle is scaled by u. Axis is offset by rotation W.

source rotation (bone)

target rotation

(surface)

We map a sequence of bone rotations to a sequence of surface rotations.We map a sequence of bone rotations to a sequence of surface rotations.

source rotation

sequence (bone)

target rotation

sequence (surface)

… …

We fit parameters u and W by regression.We fit parameters u and W by regression.

Skeleton rotations

Surface rotations u’,W’

Best-fit parameters

Rotational regression is good at capturing muscle bulges.Rotational regression is good at capturing muscle bulges.

Mesh reconstruction stitches deformation gradients together.Mesh reconstruction stitches deformation gradients together.

Deformation gradients prediction

Mesh reconstruction

Least-squares problem equivalent to linear system.

Computation is matrix-multiplication.

Mesh reconstruction solved with least-squares.Mesh reconstruction solved with least-squares.

deformation gradients

vertex positions

Least –squares

C

D(q)

Near-rigid vertices help eliminate low-frequency errors at extremities.Near-rigid vertices help eliminate low-frequency errors at extremities.

Low-frequency errors can accumulate at extremities of mesh

We fix a set of near-rigid vertices to their SSD predictions

Still a least squares problem

We build upon existing mesh reconstruction work.We build upon existing mesh reconstruction work. Mesh IK [Sumner et al. 2005], [Der et al. 2006]

SCAPE [Anguelov et al. 2005]

Similar formulation, faster evaluation.

[Anguelov et al. 2005]

Here’s a review of what we’ve covered.Here’s a review of what we’ve covered.

Rotational Regression

Deformation Gradients Prediction

Mesh Reconstruction

Least-squares problem

C

Dk(q)

Model reduction lowers the dimensionality of problem.Model reduction lowers the dimensionality of problem.

Large multiplication on CPU

Smaller multiplications on GPU

Dk(q)

C C’

Dl(q)

SSD

Model reduction uses greedy clustering.Model reduction uses greedy clustering.

Vertices clustered into proxy-bones.

Per-triangle deformation gradients clustered into “key” deformation gradients.

P = 450 225 110 50 25

Mesh reconstruction reduced to the following matrix-multiplications.Mesh reconstruction reduced to the following matrix-multiplications.

C’

Dl(q)

SSD weights

“key” deformation gradients

Map from “key” deformation gradients to proxy-bones

All on GPU:

Computation in fragment program

Skinning Mesh Animations is a an alternative approach to model reduction.Skinning Mesh Animations is a an alternative approach to model reduction.

The method from Skinning Mesh Animations uses mean-shift clustering and is more robust to errors. [James and Twigg 2005]

Our method minimizes vertex error and is faster

Deformation gradients prediction is now on “key” deformation sequences.

Deformation gradients prediction is now on “key” deformation sequences.

Fewer deformation gradient sequences to predict rotational regression.

Mesh reconstruction step reduced to matrix-multiplications on GPU.Mesh reconstruction step reduced to matrix-multiplications on GPU.

Smaller matrix-multiplications

Supported on graphics hardware

C’

Dl(q)

Our Technical Contributions:Our Technical Contributions:

Rotational Regression

Accurate and GPU-Ready Poisson Reconstruction

Model Reduction

ResultsResults

ResultsResults

ResultsResults

Our work approximates the training examples better than SSD and also generalizes well.Our work approximates the training examples better than SSD and also generalizes well.

Our model is suitable for interactive techniques.Our model is suitable for interactive techniques.

Evaluation speed within a factor of two of SSD

Off-line training preprocess is usually less than half an hour

How does our work fit with previous work?How does our work fit with previous work?

Our work is complementary to displacement correcting techniques.Our work is complementary to displacement correcting techniques.

Previous work provide corrective displacements.

– Pose space deformation [Lewis et al. 2000],

– Shape by example [Sloan et al. 2001],

– Eigenskin [Kry et al. 2002]

Our work provides better approximation of rotations.

Our work complements approaches that build upon SSD.

Figure from [Kry et al. 2001]

Displacement correcting approaches fail when SSD is very wrong.Displacement correcting approaches fail when SSD is very wrong.

Our work builds upon previous ideas on enriching the SSD model. Our work builds upon previous ideas on enriching the SSD model.

Multi-weight enveloping [Wang and Phillips 2002]

Additional joints [Mohr and Gleicher 2003]

Our technique has more parameters than SSD and generalizes the additional-bones model.

A more expressive model is useful here.A more expressive model is useful here.

Our model doesn’t do a perfect job.Our model doesn’t do a perfect job.

Not perfect reproduction

– Inspired by muscle bulging and twisting. Other behaviors empirically validated.

– Displacement correcting technique can be used for exact reproduction of examples.

Conclusion: Fast and accurate enveloping.Conclusion: Fast and accurate enveloping.

Fast evaluation of physical simulations through learning.

– Within a factor of two of SSD on most models

Accurate reproduction of details

– Better approximation and generalization

– Complementary to previous work

A replacement for linear blend skinning

AcknowledgementsAcknowledgements

Funding

– Nokia Research Center

– National Science Foundation

– Pixar Animation Studios

Hardware/Software

– NVIDIA Corporation

– Autodesk

Data

– Drago Anguelov

– Joel Anderson

– Michael Comet, Comet Digital, LLC

– Mark Snoswell, CG Character

– Joseph Teran, Ron Fedkiw

MIT Graphics Group

– Ilya Baran

– Jiawen Chen

– Sylvain Paris

Questions?Questions?

Thank you for coming to our talk!

Learning tasks trade expressiveness and simplicity.Learning tasks trade expressiveness and simplicity.

More Expressive:

Captures more types of deformation.

Simpler:

Easier to fit

Fewer training examples needed.

Less likely to overfit.

Rotational Regression

Linear blend skinning (SSD) is a rough and ready map from joint rotation matrices to vertex positions.

Linear blend skinning (SSD) is a rough and ready map from joint rotation matrices to vertex positions. Most popular enveloping technique for games

Coarse modeling parameters (but very simple)

Not expressive enough (but very fast)

desired deformation

SSD deformation

Model ReductionModel Reduction

True optimization not as tractable

We approximate it with a greedy algorithm inspired by mesh simplification.

difficult to solve simultaneously

discrete optimization

Our work builds upon previous ideas on Our work builds upon previous ideas on

Additional joints [Mohr and Gleicher 2003]

Multi-weight enveloping [Wang and Phillips 2002]

Our technique generalize the additional-

bones model

We evaluate cross-validation error to

test for overfitting

[Wang and Phillips 2002]

Rotational regression is good at capturing muscle bulges.Rotational regression is good at capturing muscle bulges.