prediction of non-linear aging trajectories of faces

Prediction of Non-Linear Aging Trajectories of Faces

K. Scherbaum, M. Sunkel, V. Blanz and H.-P. Seidel

[ 2007/5/9, Eurographics 2007, Prague ]

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Motivation / Goal

automated growth-prediction system applications

photofit-pictures of missing children automated animation, art

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

9 years

10 years 11 years

Age Progression – Optimal Case

face space

child 1

child 2

child 3

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Real Case – Support Vector Regression

face space

9 years

10 years 11 years

only 1 sample per person no longitudinal study

find isosurfaces and gradients

Runge-Kutta Integration

9 years

10 years

11 years

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Main Assumption - Curved Trajectories

use machine learning non-linear Support Vector Regression

integration of local age-gradient

growing faces transform along curved trajectories

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Challenges

learn change over time of individual facesnon-linear dependency on time, curved trajectory

learn how the change depends on individual facenon-linear dependency in face space

sparse dataset, no longitudinal study

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

3D Morphable Facemodel

System is based on a Morphable 3D Facemodel [Blanz,Vetter‘99] Built from 200 3D-face-scans of adults

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

3D Morphable Facemodel

vector space of faces

vectors with point-to-point correspondence

Shape

Texture

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Representation of Faces - Face Spaces

PCA to reduce dimensionality (yields coefficients)

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Extended Morphable Model

Extension by … plus ~238 facemodels of teenagers 3 simultaneous laser scans per face

Correspondence by … top-down approach fitting Morphable Model to new 3D faces merging original data and best fit

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Fitting the Morphable Model to 3D Scans

no optical flow because scans are often incomplete

best fit of the morphable model

merged result3D laser scans

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

learn function that maps any face x to a scalar age y

to learn this function we use …non-linear Support-Vector-Regression on training sets of l pairs

Age Progression Algorithm1

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Fitting a Regression Curve

for a given set of samples find f(x) such that all samples are within an -tube preselect and tradeoff between smoothness and errors of outliers

2

x

y

Linear: f(x) = wx + b Non-linear: f is sum of Gaussian RBF kernels K(x-xi)

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Gaussian RBF (Radial Basis Function) as kernel

we applied grid search using cross validation to optimize parameters such as (Kernelwidth) i and b are determined by SVM training

using LIBSVM for -Support Vector Regression

Non-Linear SVM Regression2

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Isosurfaces are defined in PCA space Gradient gives shortest path to next isosurface

Along the gradient … many facial changes due to aging almost no other changes

(known technique, Blanz et al. 99)

Thus: Compute growth along the gradient!

Local Aging3

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Gradient Example - Facial Attributes

gender manipulation

original

3

femalemale

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Growth Simulation: New Approach3

growth curve with given face x0 at time t currently we compute the local gradient and walk along this gradient instead we should compute the curved trajectory

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Solve differential equation … to compute curved trajectories integrate the differential equationusing Runge-Kutta algorithmperform small steps

Runge Kutta Integration4

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Visualized Aging Trajectories4

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Reducing Complexity

we did not train on all principle components speedup of SVM training we experimented with 20, 40 or 80 PCs

Justification …

growth leads to overall change of facial size significant changes are represented

by the first PCs [ large variance ] facial growth should happen in the first PCs

4

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Growth Example

growth simulation for both, shape and texture

12 14 16 18 20 years

22 24 26 28 30 years

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

More Examples

10 years

15 years

20 years

30 years

12 13 12 10 14

3D laser scans,original

age

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Background, Haircut

Face

Pose, Light

3D reconstruction and aging

Rendering the Result into Images [EG’04]

Composed Result

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Input at the age of 11

Photofit Picture Example

Possible appearances at the age of 17

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Aging in Images - Example

Picture (1999)

Ground truth pictures (2005)

Different prediction renderings

3D reconstruction and aging

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Linear vs. Non-Linear

Linear age progression perform linear regression (yields a function)

[ straight-forward least squares fit ] transform faces also along the gradient

Disadvantages … the gradient is constant [ linear function ] each face moves along

the same straight trajectory

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Linear vs. Non-Linear

comparison of age estimation error (in months) mean squared training and generalization errors

non-linear (RBF)

38.90 29.35

linear 67.66 62.87

non-linear (RBF)

32.68 18.12

linear 66.14 60.05

non-linear SVM regression behaves superior! generalization indicates: no overfitting

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Remember the Challenges

Are growth trajectories curved?Mean angle between start- and target-tangent

the trajectories are curved, not linear

10.3º 30.0º

Have different faces distinct trajectories?Mean angle of trajectories of different faces

15.7º 33.5º

the trajectories are different

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Conclusions

Results … aging involves non-linear components trajectories are distinct for different individuals linear systems are a reasonable approximation technique works without longitudinal data

But … more data would be helpful longitudinal data would allow for exact evaluation

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

MOVIE

Thank you for your attention!

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Representation of Faces - Face Spaces

arbitrary faces by linear combinations of examples

PCA to reduce dimensionality (yields coefficients)

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Main Idea … compute aging trajectories z(t) locally along gradient of the aging function f(x) and going through a start vector or face x0:

Aging Trajectories4

Kristina Scherbaum scherbaum@mpi-inf.mpg.de

Aging Information

extracted from the database of 200 adult face scans and new database of 238 face scans of teenagers

teenager overview

Kristina Scherbaum scherbaum@mpi-inf.mpg.de