2/14/00

Vision based AnimationVision based AnimationThe Inverse of an Inverse ProblemThe Inverse of an Inverse Problem

Henning BiermannChris BreglerAaron HertzmannLorie LoebKathy PullenDanny Yang

2/14/00

The Standard Vision PipelineThe Standard Vision Pipeline

Image / Video

Recognition

Modeling

Capture

2/14/00

Vision for Graphics / AnimationVision for Graphics / Animation

Image / Video

Image / Video

Synthesis

Modeling

Capture

2/14/00

Vision for Graphics / AnimationVision for Graphics / Animation

Image / Video

Expressive Animations

Modeling

Capture

2/14/00

The Standard AnalogyThe Standard Analogy

2/14/00

The Graphics ProblemThe Graphics Problem

2/14/00

The Vision ProblemThe Vision Problem

2/14/00

Vision Based AnimationVision Based Animation

2/14/00

Vision for GraphicsVision for Graphics

Image / Video

Image / Video

Synthesis

Modeling

Capture

2/14/00

Vision for GraphicsVision for Graphics

Image / Video

Image / Video

Synthesis

RicherModeling

Capture

2/14/00

Vision for AnimationVision for Animation

Image / Video

Image / Video

KinematicsDynamics

Synthesis

Capture

RicherModeling

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

Hand Coded Learned

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

Hand Coded Learned

MX Y

Analytically derived Fit Black-Box

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

• Model / Data tradeoff:

Pure Model Pure Data

Analytical Learned

2/14/00

Models: 2 IssuesModels: 2 Issues

• Explicit / Implicit tradeoff:

• Model / Data tradeoff:

Pure Model Pure Data

Analytical Learned

2/14/00

Case #1: Models of Kinematics Case #1: Models of Kinematics

- Approximate Kinematic Chains.

- Fine-tune Kinematic Chains.

- Acquisition of Nonrigid Models.

2/14/00

Approximate Kinematic Chains Approximate Kinematic Chains

- Enough for Animation ?

2/14/00

Case #1: Models of Kinematics Case #1: Models of Kinematics

- Approximate Kinematic Chains.

- Fine-tune Kinematic Chains.

- Acquisition of Nonrigid Models.

2/14/00

Fine-Tune KinematicsFine-Tune Kinematics

Anna Rabinowicz

2/14/00

Fine-Tune KinematicsFine-Tune Kinematics

Kathy Pullen

2/14/00

Case #1: Models of Kinematics Case #1: Models of Kinematics

- Approximate Kinematic Chains.

- Fine-tune Kinematic Chains.

- Acquisition of Nonrigid Models.

2/14/00

-

Acquisition of Nonrigid 3D Kinematics Acquisition of Nonrigid 3D Kinematics

With Gene Alexander, Henning Biermann, Aaron Hertzmann, Lorenzo Torresani, Danny Yang

2/14/00

Solution for Non-rigid Tracking / ReconstructionSolution for Non-rigid Tracking / Reconstruction

- Prior Model:

2D: PCA-based (Cootes et al 95) 3D: Blanz+Vetter-99

2/14/00

3D Model Acquisition3D Model Acquisition

- Multi-view input: Pighin et al 98

2/14/00

Solution for Rigid 3D AcquisitionSolution for Rigid 3D Acquisition

Structure from Motion:

- Tomasi-Kanade-92

Factorization

3D Pose3D rigid Object

2/14/00

Acquisition without prior model ?Acquisition without prior model ?

• No Model available ?

• Model too generic/specific ?

• Stock-Footage only in 2D ?

2/14/00

Solution based on Factorization Solution based on Factorization

- We want 3 things:- 3D non-rigid shape model- for each frame:

- 3D Pose- non-rigid configuration (deformation)

-> Tomasi-Kanade-92:

W = P S

Rank 3

2/14/00

Solution based on Factorization Solution based on Factorization

- We want 3 things:- 3D non-rigid shape model- for each frame:

- 3D Pose- non-rigid configuration (deformation)

-> PCA-based representations:

W = P non-rigid S

Rank K

2/14/00

Factoring twice Factoring twice

-

2/14/00

Nonrigid 3D Kinematics from point tracks Nonrigid 3D Kinematics from point tracks

-

2/14/00

-

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

2/14/00

-

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

2/14/00

-

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

2/14/00

Motion Capture

Modeling

Synthesis

Nonrigid 3D Kinematics from dense flow Nonrigid 3D Kinematics from dense flow

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Learning Controller Mocap. Library

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Parametric Non-Parametric

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Pure Model Pure Data

Kernel-Based Densities

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

Kathy Pullen

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

“Motion Texture”

Sampling /Matching

2/14/00

Case #2: Models of Dynamics Case #2: Models of Dynamics

P(Li | Lj)

Sampling / Matching:

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

2/14/00

Synthesizing Synthesizing Subtleties + VariationsSubtleties + Variations

Kathy Pullen

Motion Capture

Modeling

Synthesis

2/14/00

Vision Based AnimationVision Based Animation

Animations

KinematicsDynamics

RicherModeling

2/14/00

Human Movement ResearchHuman Movement Research

Computer Science:Vision / Graphics

Bio-Mechanics:Motion Models/Analysis

Art:Animation / Dance

Medicine/ Neurology:Movement Disorders

2/14/00

Model

Data

2/14/00

Model

Data

Expressive AnimationExpressive Animation

2/14/00

Human Movement ResearchHuman Movement Research

Computer Science:Vision / Graphics

Bio-Mechanics:Motion Models/Analysis

Art:Animation / Dance

Medicine/ Neurology:Movement Disorders

2/14/00

Project Mariposa: Animation PrinciplesProject Mariposa: Animation Principles

Lorie Loeb

2/14/00

Project Mariposa: Animation PrinciplesProject Mariposa: Animation Principles

Lorie Loeb

Realism is out

2/14/00

Project Mariposa: Animation PrinciplesProject Mariposa: Animation Principles

Lorie Loeb

Realism is out

Expressiveness is in

2/14/00

Model

Data

Expressive AnimationExpressive Animation

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

2/14/00

Model

Data

Controversy: Motion-Capture based AnimationControversy: Motion-Capture based Animation

2/14/00

Synthesis by AnalysisSynthesis by Analysis

Animations

2/14/00

Enough for Animation ?Enough for Animation ?

Expressive Animations

Video/Mocap

2/14/00

Expressive AnimationExpressive Animation

Expressive Output

CartoonCapture