motion editing and retargetting
DESCRIPTION
Motion Editing and Retargetting. Jinxiang Chai. Outline. Motion editing [video, click here ] Motion retargeting [video, click here ]. Required Readings. Comparing Constraint-Based Motion Editing Methods Retargeting Motion to New Characters . Kinematic Motion Editing. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/1.jpg)
Motion Editing and Retargetting
Jinxiang Chai
![Page 2: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/2.jpg)
Outline
• Motion editing [video, click here]
• Motion retargeting [video, click here]
![Page 3: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/3.jpg)
Required Readings
• Comparing Constraint-Based Motion Editing Methods
• Retargeting Motion to New Characters
![Page 4: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/4.jpg)
Kinematic Motion Editing
• Goal: Edit an input motion sequence to achieve new goals specified by the user
![Page 5: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/5.jpg)
Kinematic Motion Editing
• Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user
e.g., edit the motion to meet new constraints
![Page 6: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/6.jpg)
Kinematic Motion Editing
• Goal: Edit an input motion sequence to satisfy achieve new goals specified by the user
e.g., edit the motion to meet new constraints- could be multiple constraints
![Page 7: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/7.jpg)
Toy Examples
• Edit 1D function to match user constraints
Old motion M={m(t)| t=1,…,T}
t
m(t)
![Page 8: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/8.jpg)
Toy Examples
• Edit 1D function to match user constraints
t
m(t)
Old motion M={m(t)| t=1,…,T}
![Page 9: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/9.jpg)
Toy Examples
• Edit 1D function to match user constraints - could be multiple constraints
t
m(t)
Old motion M={m(t)| t=1,…,T}
![Page 10: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/10.jpg)
Toy Examples
• Edit 1D function to match user constraints - could be multiple constraints
Old motion m(t), t=1,…,T
t
m(t)
So how can we generate a new motion M’ to satisfy new constraints c?
![Page 11: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/11.jpg)
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
M(t)
Old motion m(t), t=1,…,T
![Page 12: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/12.jpg)
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
m’(t)
Does this work?
![Page 13: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/13.jpg)
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
m’(t)
Does this work? - probably not, the new motion is not smooth!
![Page 14: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/14.jpg)
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
t
M’(t)
Does this work? - probably not, the new motion is not smooth! - filter the motion!
![Page 15: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/15.jpg)
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
• Filter the edited motion M’
t
M’(t)
Does this work? - probably no, the new motion is not smooth! - filter the motion! - but the new motion now does not meet cons!
![Page 16: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/16.jpg)
Toy Examples: Idea #1
• Modify the motion m(t), t=1,…,T using new constraints C
• Filter the edited motion M’
t
M’(t)
Does this work? - probably no, the new motion is not smooth! - filter the motion! - repeat modification and filtering process!
![Page 17: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/17.jpg)
Idea #1: Modification & Filter
![Page 18: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/18.jpg)
Another Example: Three Constraints
![Page 19: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/19.jpg)
Another Example: Three Constraints
![Page 20: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/20.jpg)
Idea #1: Summary
• Pros - easy to implement - computationally efficient• Cons - does not handle complex constraints, e.g., constraint
functions involved more than one frames
Old motion M={m(t)| t=1,…,T}
t
m(t)
t1 t2
e.g., m’(t1)-m’(t2)=5
![Page 21: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/21.jpg)
Idea #1: Summary
• Pros - easy to implement - computationally efficient• Cons - does not handle complex constraints, e.g., constraint
functions involved more than one frames
How to address this limitation?
![Page 22: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/22.jpg)
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
![Page 23: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/23.jpg)
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C
![Page 24: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/24.jpg)
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
But this is an ill-posed problem - There are tons of solutions that satisfy the new constraints C - Remove the ambiguity by picking the solution with a minimal change across an entire motion
![Page 25: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/25.jpg)
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
2
,...,1),(' )()('minarg tmtmTttm
CMf )(
![Page 26: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/26.jpg)
Idea #2: Constrained Optimization
• Modify the motion M to satisfy new constraints C
Old motion m(t), t=1,…,T
t
m(t)
2
,...,1),(' )()('minarg tmtmTttm
CMf )(
Minimize the changes
Satisfy new constraints
![Page 27: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/27.jpg)
Idea #2: Constrained Optimization
• We can also formulate this as an unconstrained optimization problem
Old motion m(t), t=1,…,T
t
m(t)
22
,...,1),(' )()()('minarg CMftmtmTttm
![Page 28: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/28.jpg)
Idea #2: Constrained Optimization
• This is also called as spacetime optimization or trajectory optimization
• Simultaneously compute the entire motion
Old motion m(t), t=1,…,T
t
m(t)
22
,...,1),(' )()()('minarg CMftmtmTttm
![Page 29: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/29.jpg)
Idea #2: Constrained Optimization
• This is also called as spacetime optimization or trajectory optimization
• Simultaneously compute the entire motion
Old motion m(t), t=1,…,T
t
m(t)
22
,...,1),(' )()()('minarg CMftmtmTttm
![Page 30: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/30.jpg)
Now Back to Human Motion Editing!
![Page 31: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/31.jpg)
A sequence of poses: q1,q2,…qT
Each pose is represented as a high-dimensional vector qt: Rn
Motion trajectoriesPose qt Motion q1,…qT
Human motion representation
![Page 32: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/32.jpg)
Human Motion Editing
• Human motion is a high-dimensional function
• Constraints could be any kinematic constraints
![Page 33: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/33.jpg)
User Constraints
Any kinematic constraints throughout the motion
Position cons.
Orientation cons.
Distance cons.
Joint angle cons.
![Page 34: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/34.jpg)
Solution #1: Per-frame IK+Filtering
• Step 1: Per-frame inverse kinematics to modify poses in each frame with new constraints.
- this might be ill-posed - remove ambiguity by minimizing the difference between old
poses and new poses
• Step 2: motion filtering.
• Repeat step 1 and 2 until the solution converges
![Page 35: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/35.jpg)
Solution #2: Spacetime Motion Editing
• Consider all constraints simultaneously
• Use optimization to solve the entire motion
22
,...,1),(' )()()('minarg CMftmtmTttm
![Page 36: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/36.jpg)
Solution #2: Spacetime Motion Editing
• Consider all constraints simultaneously
• Use optimization to solve the entire motion
22
,...,1),(' )()()('minarg CMftmtmTttm
- Currently, minimizing joint angle poses across the entire sequence- A better criterion is possible!
![Page 37: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/37.jpg)
Solution #2: Spacetime Editing
• Better Criterion? - Geometric constraints such as contact constraints are often
more important
![Page 38: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/38.jpg)
Solution #2: Spacetime Editing
• Better Criterion? - Frequency Content or details
![Page 39: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/39.jpg)
Solution #2: Spacetime Motion Editing
22
,...,1),(' )()()('minarg CMftmtmTttm
Implementation details:
- use Spline representation to reduce the solution space
- often computationally expensive
- optimization over sub windows when working on a long motion sequence
![Page 40: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/40.jpg)
Motion Retargeting
• One motion, a cast of characters
• Focus on similar structure
![Page 41: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/41.jpg)
• Formulated as motion editing problem
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Skeleton of new characters
![Page 42: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/42.jpg)
• Step 1: Extract constraints/properties C from source motion
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Extracted constraints
![Page 43: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/43.jpg)
• Step 1: Extract constraints/properties C from source motion
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Extracted constraintsRetargeted motion Source motion
![Page 44: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/44.jpg)
• Step 1: Extract constraints/properties C from source motion
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
Extracted constraintsRetargeted motion Source motion
Skeleton of new characters
![Page 45: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/45.jpg)
• Step 2: Apply to new characters
Motion Retargeting: Key Idea
![Page 46: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/46.jpg)
• Step 3: Approximate answer
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;
How to initialize m’(t)?
![Page 47: Motion Editing and Retargetting](https://reader036.vdocuments.us/reader036/viewer/2022062721/568135b9550346895d9d1fc2/html5/thumbnails/47.jpg)
• Step 4: Solve constraints using spacetime optimization
Motion Retargeting: Key Idea
22
,...,1),(' )()()('minarg CsMftmtm newTttm ;