rick parent - cis681 motion capture use digitized motion to animate a character

Rick Parent - CIS681

Motion Capture

Use digitized motion to animate a character

Rick Parent - CIS681

Animation Techniques

Digitize motion

Procedural - simuleateModel or rulesSet initial conditions

KeyframeSurface model

Motion librariesStitch togethere.g., games

Rick Parent - CIS681

TechnologiesPassive optical - reflective markers

Multiple camerasNeed non-reflective environ.

Active optical - LEDs pulse IDNeed power supplyMultiple cameras

Magnetic - Active sensors CheaperNeed magnetic-neutral environmentNeed power supplyTether or wireless comm.Smaller work area

Mechanical - RotaryOnly posesCheapest

Rick Parent - CIS681

Magnetic

Up to 144Hz

13-18 6 DoF sensors

Rick Parent - CIS681

Passive Optical Motion Capture

Rick Parent - CIS681

Active Optical Motion Capture

Rick Parent - CIS681

Optical Motion Capture

Instrument “talent”

Compute joint positions from 3D positions of multiple surface markers

Convert multiple 2D marker positions to 3D positionsEstablish correspondence of markers in imagesTriangulate 3D position using two or more images

Create synthetic character with those dimensions and animation

Determine limb lengths from joint coordinates

Capture 2D position of markers in multiple cameras

Rick Parent - CIS681

Hand Motion Capture

Movement too intricate for opticalTypes of sensors

MechanicalFiber optic

Rick Parent - CIS681

Facial Motion Capture

Motion too detailed for sensing at distance

Rick Parent - CIS681

Motion Capture Procedure

1. Plan

2. Capture

3. Clean

4. Edit

5. Map

Rick Parent - CIS681

ProblemsDimensions of instrumented talent has to match synthetic charcter

Motion captured is what you get - period

Limitations on spatial extent of motion

Restricted movement because of instrumentation (harness, power supply, physical attachments, etc.)

Limitation on complexity of motion because of sensors(e.g., severe occlusions, etc.)

Rick Parent - CIS681

ResearchBlending

Blending from one motion to anotherSignal processingSpace-time constriants

RetargetingModify angles based on limb lengthsMaintain constraints (e.g. foot on floor)

AdaptingModify secondary DoFse.g. wave arm while walking

Rick Parent - CIS681

Motion WarpingMotion curves

Sparse keyframe-like constraintsKeep similar to originalWarp each curve independently

Constraints (i,ti)

Time warp constraints (t’j,tj)

’(t’) defined by ’(t) = f(,t)t= g(t’) g(t’): smooth, well-behaved

Rick Parent - CIS681

Motion Warping

’(t) = a(t)(t) + b(t)

User specifies amounts of a(ti) and b(ti)

Use interpolating spline to define a(t) and b(t) over length of curve

t

(t)

Rick Parent - CIS681

Motion Warping - time warp

t’

t

t

(t)

Stretch and compress to overlapUse interpolating spline

Linearly interpolate(t) = (t) + (1-) (t)

Rick Parent - CIS681

Motion Warping

Rick Parent - CIS681

Motion Warping

Rick Parent - CIS681

Motion Warping

Rick Parent - CIS681

Motion Warping

Rick Parent - CIS681

Motion EditingMultiresolution filteringCascade of lowpass filters

Convolve w/ B-spline 5x5 filter kernelSubsample image by factor of 2Until 1 pixel - DC component

Bandpass pyramidRepeatedly differencing 2 successive lowpass imagesExpand subtracted imagesImage reconstruction: add up all bandpass plus DC

Rick Parent - CIS681

Motion EditingMotion

Low frequency; general gross motionHigh frequency; detail, subtleties, noiseEach motion parameter => signalCalculate lowpass & bandpassOver # frames by expanding filter

LowpassG0: solidG3: dashed

BandpassL0: solidL2: dashed

Rick Parent - CIS681

Motion Editing

1. Calculate lowpass sequence

2. Calculate bandpass filter bands

3. Adjust gains and multiply Lk’s by gain values

4. Blend bands5. Reconstruct signal

Rick Parent - CIS681

Motion Editing

Adjusting gains of bands for joint positions

Rick Parent - CIS681

Motion Editing

Multitarget interpolation (like Motion Warping)

Rick Parent - CIS681

Motion Editing

Multitarget motion interpolation using frequency bands

Rick Parent - CIS681

Motion Editing

Blending between two walks without (top) and with (bottom) correspondence in time

Rick Parent - CIS681

Motion Editing

Blending two waves without (top) and with (bottom) corresponence in time

Rick Parent - CIS681

Motion Editing

1. Keyframes2. Interpolate3. Adjust4. Recalculate5. Apply

Rick Parent - CIS681

Motion Editing

Example of applying displacement curves

Rick Parent - CIS681

Retargeting

Assume identical structure

Use motion displacementsm(t) = m0(t) + d(t)

IK to enforce constraints can add High-frequence components

Use spacetime constraints

Rick Parent - CIS681

Retargeting

Minimize difference from original motion

Constraints (kinematic)Joint limitsFootplantsPoint at same placePoint on character follows another point2 points at specified distanceVector between points at specified orientation

Rick Parent - CIS681

Spacetime Constraints

Consider spacetime particlePosition: x(t)Jet-force: f(t)Motion equation: mx -f - mg = 0

Constraintsx(t0) = ax(t1) = b

Minimize

€

R =2

f (t) dtt0

t1

∫

Find f such that x satisfies the boundary conditions andR is minimized

Numerical solution: sequential quadratic programming

Rick Parent - CIS681

Sequential Quadratic Programming

Compute second order Newton-Raphson step in RCompute first order Newton-Raphson step in the C’sCombine by projecting the first onto the null space of the second

Requires Jacobian of the constraint function and Hessian of the objective function

Final update = Sj + Tj

Until C’s = 0 and R at minimum

€

−∂R∂Si

= H ijS jj

∑

€

−Ci = Jij (S j + Tj )∑

Rick Parent - CIS681

Linear System Solving

Use pseudo inverse

Use conjugate gradient to compute the pseudo inverse

Rick Parent - CIS681

Retargeting

Rick Parent - CIS681

Retargeting

Rick Parent - CIS681

Retargeting

Rick Parent - CIS681

Retargeting