reach out and touch space (motion learning)
DESCRIPTION
Reach Out and Touch Space (Motion Learning). Luis Goncalves, Enrico Di Bernardo, Pietro Perona California Institute of Technology. Motion is an important cue. A system for body tracking (Goncalves-Di Bernardo-Perona, ICCV ‘95). Single camera Real-time estimate of body pose 3D Model-based - PowerPoint PPT PresentationTRANSCRIPT
Reach Out and Touch Space(Motion Learning)
Luis Goncalves, Enrico Di Bernardo, Pietro Perona
California Institute of Technology
Motion is an important cue
A system for body tracking(Goncalves-Di Bernardo-Perona, ICCV ‘95)
• Single camera
• Real-time estimate of body pose
• 3D Model-based
• No markers required
• Able to track at frame rate (30 fr/sec)
• 8% max error (along the line of sight)
• No loose clothing• Calibration on the user • Loses track for fast movements
BUT
A system for body tracking(Goncalves-Di Bernardo-Perona, ICCV ‘95)
Arm silhouette generation 3D Model
BackgroundSubtraction
Camera Errorvector
Estimated arm position and velocity
RecursiveEstimator
Calibrationparameters
Arm modelparameters
DynamicalModel
Current model: Random Walk
Dynamical model equations:
),0(, Νwwv
v
Why Models for Human Motion ?
• Locomotion (biomechanics, robotics)
• Brain motor control (neuroscience)
• Human/Machine perception of biological
motion (neuroscience, psychophysics, computer vision)
• Realistic animation (computer graphics)
Invariant properties of ballistic point to point movements
• The path is approximately straight
• The tangential velocity profile has a smooth bell shape
• These properties are invariant wrt subject, execution time, load carried
(Hollerbach, Viviani, Flash-Hogan, Bizzi)
Far from the neuro-muscular limits and after practice:
No predictive power for general motion
Human figure animation in CG
• Keyframing, manual editing (Perlin, animation software)
• Physics-based (Hodgins et al.)
• Constraint optimization (Witkin and Kass, Badler et al.)
• Human motion capture (Bruderlin, Rose et al.)
The proposed method
• Acquire sample human motions
• Label each individual motion with a high level description (e.g. goal of motion)
• Learn a function that maps labels to motions
The 2D Motion Capture System
• 14 dots on the main body joints• single camera• UV lighting• real-time detection with sub-pixel accuracy
Title:/home2/dibe/matlab/MLD/reports/MLDibe.psCreator:XV Version 3.10 Rev: 12/16/94 - by John BradleyPreview:This EPS picture was not savedwith a preview included in it.Comment:This EPS picture will print to aPostScript printer, but not toother types of printers.
Example: Reaching motions
Start from a fixed initial pose.
Reach to various locations in space.
Pictorial representationLabel space:
Trajectory space:
LNLlTN*28Mm
The Functional Space
NLdN
ddj L
ll 10
11)( l1. Polynomial Basis Functions
2. Radial Basis Functions
T
jjj
ej
)()(21 1
)(μlΣμl
l
Learning the function
ki
ik
ik
wjk mfw
jk ,
2
}{))((arg min l
ΦMW
N
jjjkkk wfm
0)()(ˆ ll
Experiments: Reaching motions
• NL = 2
• NT = 120
Picking up apples in30 different locations
90 sample motions
Experiments: Drawing motionsDrawing strokes on a blackboard• NL = 8
• NT = 60110 sample strokes
How to evaluate performance?
• RMS error
• Perceptual evaluation by subjects
• other methods ...
Results: %RMS error for Reaching
Results: Visual DiscriminabilityReaching using 3rd order polynomial functional space
Results: %RMS error for Drawing
Results: Visual DiscriminabilityDrawing using 1st order polynomial functional space
Combining Reaching and Drawing
Blend out trajectory discontinuities at merge points of different motions.
Conclusions
• The method generates realistic synthetic reaching and drawing motions • The method can generate motions from a high level description
• The technique can be used for animation
Future work• Obtaining and running experiments with 3D captured data
• Develop a perceptually motivated metric
• Experiment with other high level labels such as speed, emotional state or gender
• Use models for tracking/prediction
Minimum jerk trajectories in reaching movements (Flash-Hogan ‘85)
Explains the experimental evidence for both straight and via point reaching motions
The trajectory minimizes:
Tt
t
dttC0
2)(x
The equilibrium point trajectory (Bizzi ‘91)
The muscular system has a spring-like behavior
Brain signals activate entire muscle groups
The activation signals depend only on the ideal (minimum jerk) trajectory
Motion is a powerful cue
From the trajectories of 12 dots attached to the main joints in the body, a subject could• distinguish human motion from other objects motion• identify gender• identify action and mood• perceive 3D structure from 2D trajectories(Johansson ‘73, Mather ‘94, Dittrich et al. ‘96)
Labeling the data