pose space deformation - ethz · pose space deformation body animation 1. pose space ... –no poes...
TRANSCRIPT
Simone Croci
1
Pose Space DeformationA unified Approach to Shape Interpolation and Skeleton-Driven
Deformation
J.P. Lewis Matt Cordner Nickson Fong
Presented by Simone Croci
Simone Croci
2
Talk Outline• Character Animation
OverviewProblem Statement
• BackgroundSkeleton-Subspace DeformationShape Interpolation
• Pose Space DeformationDeformation ModelEvaluationRelated Works
Simone Croci
3
Character Animation
OverviewAnimation:
“To give a soul to a lifeless character”
Simone Croci
4
Character Animation
Problem StatementMain Components:
1. Character Model
2. Deformation model
Simone Croci
5
Character Animation
Deformation Model
Complexity:∀ vertex: 3 DoFs
=> Reduction of DoFsx
y
z
Simone Croci
6
Character Animation
Deformation Model
Mapping:Parameters => Deformation
Simone Croci
7
Character Animation
Deformation ModelJoint rotations of a skeleton
Simone Croci
8
Character Animation
Deformation ModelEmotional Axes
X=Sad/Happy Y=Relaxed/ Excited
Simone Croci
9
Character Animation
Deformation ModelReduction of DoFs
ReducedDeformation Freedom
MovementDeficiencies
Lack of realism
Difficulty tocontrol the deformations
PathologicalDefects
Simone Croci
10
Character Animation
Deformation ModelReduction of DoFs
ReducedDeformation Freedom
solves these problems
retaining few DoFs
MovementDeficiencies
Lack of realism
Difficulty tocontrol the deformations
PathologicalDefects
Pose Space Deformation
Simone Croci
11
Skeleton-Subspace Deformation
…also called skinning, enveloping
Deformation ModelSkeleton => Vertex Position
Simone Croci
12
Skeleton-Subspace Deformation
Initial posey
x
v
v’’
v’1. Local
coordinatesv’,v’’
2. Vertex-joint weights: w`, w``
Simone Croci
13
Skeleton-Subspace Deformation
New posey
x
v
v = w` T’(v`)+ w`` T’’(v``)
v’v’’
T from local toworld coordinates
Simone Croci
14
Skeleton-Subspace Deformation
Pros• Simple• Smooth deformations• Low memory requirements• Real-time deformations
Simone Croci
15
Skeleton-Subspace Deformation
Cons• Indirect control on deformations through
weights
• Deformation Subspace is limited=> Pathological Defects (complex Joint
configurations, “Collapsing Elbow”)
Simone Croci
16
Skeleton-Subspace Deformation
Collapsing Elbow
Simone Croci
17
Skeleton-Subspace Deformation
Frame 1 Frame 2
Initial pose
Simone Croci
18
Skeleton-Subspace Deformation
Frame 1
New pose
Frame 2
Simone Croci
19
Skeleton-Subspace Deformation
Cons• Indirect control on deformations through weights
PSD: direct sculpting
• Deformation Subspace is limited=> Pathological Defects (complex Joints, “Collapsing Elbow”)
PSD: interpolation of a vast range of deformations
Simone Croci
20
Shape Interpolation
∑=k
kkSwS
also called shape blending, multi-target morphing
AlgorithmInput: key shapes Sk
(same topology)
SuperpositionAnger Disgust Fear
Joy Sadness Surprise
Simone Croci
21
Shape Interpolation
neutral smirk
+ =>
half smirk
Simone Croci
22
Shape Interpolation
smirk
+ )=½(half smirkneutral
Simone Croci
23
Shape Interpolation
smirk
+ )=½(half smirkneutral
Simone Croci
24
Tony de Peltrie1985
Simone Croci
25
Shape Interpolation
Pros• Direct manipulation of desired expressions• Skin deformation for facial animation
(Used in the movies)
Simone Croci
26
Shape Interpolation
Cons• Not suited for skeleton-based deformations
Simone Croci
27
Shape Interpolation
Cons• Not suited for skeleton-based deformations• Storage expensive
Simone Croci
28
Shape Interpolation
Cons• Not suited for skeleton-based deformations• Storage expensive• Conflicting Key Shapes
Simone Croci
29
Shape Interpolation
Conflicting Key Shapes
Key Shapes are not independent
Superpositiona KSHappy+ b KSSurprise
Simone Croci
30
Shape Interpolation
Conflicting Key Shapes
Key Shapes are not independent
Superpositiona KSHappy+ b KSSurprise + c KSFear
Simone Croci
31
Shape Interpolation
Conflicting Key Shapes
Key Shapes are not independent
Superpositiona’ KSHappy+ b’ KSSurprise + c KSFear
Simone Croci
32
Shape Interpolation
Conflicting Key Shapes
Key Shapes are not independent
Superpositiona’ KSHappy+ b’ KSSurprise + c KSFear
PSD: interpolation between key Shapes
Simone Croci
33
Shape Interpolation
Cons• Not suited for skeleton-based deformations• Storage expensive• Conflicting Key Shapes• Key Shape <=> Animation Parameter
Simone Croci
34
Shape Interpolation
Key Shape Anim. Parameter
If new expression is needed=> New Key shape & New Parameter
∑=k
kkSwS
Simone Croci
35
Shape Interpolation
Key Shape Anim. Parameter
If new expression is needed=> New Key shape & New Parameter
PSD: no additional parameter
freedom in the design of parameter space
∑=k
kkSwS
Simone Croci
36
Comparison TablePose Space Deformation
Low requirementsHigh requirementsMemory
YesNot alwaysSmoothness of Interpolation
NoYesDirect manipulation
Real-timeReal-timePerformance
Body (skeleton-influenced) Deformation
Facial Deformation
Field of application
Skeleton-subspace Deformation
Shape Interpolation
Simone Croci
37
Comparison Table
Real-time
Pose Space Deformation
Low requirementsHigh requirementsMemory
YesNot alwaysSmoothness of Interpolation
NoYesDirect manipulation
Real-timeReal-timePerformance
Body (skeleton-influenced) Deformation
Facial Deformation
Field of application
Skeleton-subspace Deformation
Shape Interpolation
Simone Croci
38
Comparison Table
Low requirements
Real-time
Pose Space Deformation
Low requirementsHigh requirementsMemory
YesNot alwaysSmoothness of Interpolation
NoYesDirect manipulation
Real-timeReal-timePerformance
Body (skeleton-influenced) Deformation
Facial Deformation
Field of application
Skeleton-subspace Deformation
Shape Interpolation
Simone Croci
39
Comparison Table
If needed
Low requirements
Real-time
Pose Space Deformation
Low requirementsHigh requirementsMemory
YesNot alwaysSmoothness of Interpolation
NoYesDirect manipulation
Real-timeReal-timePerformance
Body (skeleton-influenced) Deformation
Facial Deformation
Field of application
Skeleton-subspace Deformation
Shape Interpolation
Simone Croci
40
Comparison Table
Yes
If needed
Low requirements
Real-time
Pose Space Deformation
Low requirementsHigh requirementsMemory
YesNot alwaysSmoothness of Interpolation
NoYesDirect manipulation
Real-timeReal-timePerformance
Body (skeleton-influenced) Deformation
Facial Deformation
Field of application
Skeleton-subspace Deformation
Shape Interpolation
Simone Croci
41
Comparison Table
Facial and Body Deformations
Yes
If needed
Low requirements
Real-time
Pose Space Deformation
Low requirementsHigh requirementsMemory
YesNot alwaysSmoothness of Interpolation
NoYesDirect manipulation
Real-timeReal-timePerformance
Body (skeleton-influenced) Deformation
Facial Deformation
Field of application
Skeleton-subspace Deformation
Shape Interpolation
Simone Croci
42
Character Animation
Deformation Model
Mapping:Parameters => Deformation
Simone Croci
43
Character Animation
Deformation Model
Mapping:Parameters => Deformation
Interpolat from deformation examples
Pose Space Deformation
Simone Croci
44
Character Animation
Deformation Model
Mapping:Parameters => DeformationPose Space => Displacements Δx
Pose Space Deformation
Interpolat from deformation examples
Simone Croci
45
Pose Space Deformation
Face Animation1. Pose Space
Ex: Emotion Space
2. Initial Face Model
3. Sculpting of exampleswith Pose Space Coordinates[Δx: between initial model and examples]
4. Interpolant
Simone Croci
46
Pose Space Deformation
Face Animation
Simone Croci
47
Pose Space Deformation
Body Animation1. Pose Space
Ex: Skeleton Parameters
2. Skeleton & Model
3. Sculpting of exampleswith Pose Space Coordinates[Δx: between SSD and examples]
4. Interpolant
SSD PSD
Simone Croci
48
Pose Space Deformation
Body Animation
Simone Croci
49
Interpolat from ExamplesInterpolant:
Pose Space => Displacements Δxfrom {(PSCoord1, Δx1), ..., (PSCoordN, ΔxN)}
Scattered Data Interpolation:Interpolation of a set of irregularly located data points
Approaches:- Shepard’s Method- Radial Basis Function Interpolation
Simone Croci
50
Shepard’s Method
are the given data points at
0,)( >−= − pw pkk xxx
∑
∑
=
== N
kk
N
kkk
w
dwd
1
1
)(
)()(ˆ
x
xx
Ndd ,,1 K Nxx ,,1 K
Simone Croci
51
Weight function
Simone Croci
52
Flat zones
Simone Croci
53
Average at far points
Simone Croci
54
Shepard’s Method
Properties:• Singular at data points ( )• Infinitely differentiable except at data points• Partial derivatives are zero at data points
=> flat zones around data points• Far from data points converges to the
average value
kx
N
d
w
dwd
N
kk
N
kk
N
kkk ∑
∑
∑=
=
= =∞
∞=∞ 1
1
1
)(
)()(ˆ
0with )( >−= − pw pkk xxx
kx
Simone Croci
55
Radial Basis Functions…also called Hardy’s multiquadratics
from at solving a system of linear equations:
( )∑=
=N
kkkwd
1-)(ˆ xxx ψ
Nww ,,1 K Ndd ,,1 K Nxx ,,1 K
( ) ( )
( ) ( ) ⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛=
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
NNNN
Nk
N w
w
d
dM
L
MOM
L
M1
1
111
--
--
xxxx
xxxx
ψψ
ψψ
wd Ψ=
Simone Croci
56
Choice of Radial Basis Function
( ) rr =ψ
kr xx -=
( ) 0,)( 22 >+= − ασψ αrr
( ) )ln(2 rrr =ψ
( ) 10,)( 22 <<+= βσψ βrr
( ) 3rr =ψ
localized
Thin-plate spline function
linear
cubic
Simone Croci
57
Gaussian Radial Basis
=>
( ) ⎟⎟⎠
⎞⎜⎜⎝
⎛ −= 2
2
2exp
σψ rr
kr xx -=
∑= ⎟
⎟
⎠
⎞
⎜⎜
⎝
⎛ −=
N
k
kkwd
12
2
2-
exp)(ˆσxx
x
Simone Croci
58
Gaussian Basis Function
Simone Croci
59
Gaussian Radial Basis
Simone Croci
60
Gaussian Radial Basis
Properties:- smooth- localized:
=> width of falloff is adjustable- relatively fast to compute (table)
( ) ∞→= rr for0ψσ
( ) ⎟⎟⎠
⎞⎜⎜⎝
⎛ −= 2
2
2exp
σψ rr
Simone Croci
61
Pose Space Deformation[Summary]
∑=
=N
kkwd
1)()(ˆ xx ψ
Deformation model:Interpolant:
Pose Space => Displacements Δx
from {(PSCoord1, Δx1), ..., (PSCoordN, ΔxN)}
Simone Croci
62
Workflow
l
Simone Croci
63
PerformanceFor N poses:
• Preprocessing phase:- NxN Matrix must be inverted- Matrix-vector multiplication
(∀ component of ∀ displaced vertex)
• Animation phase:- Interpolated table lookup
wd Ψ=
dw 1−Ψ=Ψ
( )∑=
=N
kkkwd
1-)(ˆ xxx ψ
( ) ( )
( ) ( )⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛=Ψ
NNN
Nk
xxxx
xxxx
--
--
1
11
ψψ
ψψ
L
MOM
L
Simone Croci
64
Memory Requirements
For N poses:Every vertex stores Nx3 weights
(N weights ∀ component of a vertex)N pose space coordinates of ex.
( )∑=
=N
kkkwd
1-)(ˆ xxx ψ
Simone Croci
65
Memory Requirements
For N poses:Every vertex stores Nx3 weights
(N weights ∀ component of a vertex)N pose space coordinates of ex.
( )∑=
=N
kkkwd
1-)(ˆ xxx ψ
Simone Croci
66
Memory Requirements
For N poses:Every vertex stores Nx3 weights
(N weights ∀ component of a vertex)N pose space coordinates of ex.
( )∑=
=N
kkkwd
1-)(ˆ xxx ψ
Simone Croci
67
Pros & ConsPros:• Wide range of deformations• Arbitrary Pose Space Axes• Real-time synthesis • Relatively simple to implement• Control on interpolation ( )• Direct manipulation
(Iterative layered refinement)
σ
Simone Croci
68
Pros & Cons
Cons:• Accuracy is reliant on the modeler/animator• is manually tuned• Performance• Memory requirements
σ
Simone Croci
69
Critiques
• Least Square Problem ?with regular
• If N poses then 3 NxN matrices must be inverted for each control vertex?
Only one NxN matrix must be inverted• Close coordinates in PS results in a
numerically unstable matrix (regularization, TSVD)
dΨw 1−=( ) dΨΨΨw TT 1−
=Ψ
Ψ
Ψ
Simone Croci
70
Related Papers
Shape from ExamplesSloan, Rose, Cohen (I3D conference 2001)
EigenSkinKry, James, Pai (SIGGRAPH 2002)
Skinning Mesh AnimationJames, Twigg (SIGGRAPH 2005)
Simone Croci
71
Shape by ExamplesSloan, Rose, Cohen
• Paradigm: Design by Examples• Interpolation in Lagrangian form• Radial Basis Functions (B-Splines) and Polynomials• Interpolation & Extrapolation• Reparameterization• Applied also to Textures• Preprocessing phase:
– one linear system per example• Animation phase: twice faster
Simone Croci
72
Shape by ExamplesSloan, Rose, Cohen
Simone Croci
73
EigenSkinKry, James, Pai
• Paradigm: Design by Examples• Articulated characters• Deformation field for each Joint support
– EigenDeformations (Deformation basis)• Mapping: Joint => Eigendeformation coordinates
– 1-D interpolation with Radial Basis Functions– No non-linear joint-joint coupling effects
• Graphical Hardware Optimization– Reduction of Memory Requirements– Real-time rendering
Simone Croci
74
EigenSkinKry, James, Pai
Simone Croci
75
Skinning Mesh AnimationJames, Twigg
• Approximation of sequence of input meshes– No Pose Space
• Skinning algorithm:– Proxy bone Transformations (+ weights)
automatically approximated– Displacement field in rest pose (TSVD)
• Real-time rendering (Graphical HW support)
... more to come
Simone Croci
76
Question Time