coregistration: simultaneous alignment and modeling of

The Optimization By minimizing a single objective function, we reliably obtain high quality alignments to noisy scans while simultaneously learning a realistic articulated body model. Coregistration alternates between updating each alignment T k and pose k , each person’s body shape D , and the global pose deformation model Q(). Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape David A. Hirshberg Matthew Loper Eric Rachlin Michael J. Black MPI for Intelligent Systems, Perceiving Systems Department The Corpus Registration Problem Goal: Align an entire corpus of body scans. Build a highly realistic 3D body model. Solution: Coregistration simultaneously registers the corpus and builds the model. Challenge Register a 3D template mesh to each scan. Each aligned template must closely match the surface of each scan. All alignments must also be in accurate point-to-point correspondence. Goal An accurate 3D model of the body that captures non-rigid pose deformation and individual shape variations across a population. Outputs When coregistering a corpus of scans we obtain: • T k : An alignment to each scan S k • k : A pose estimate for each scan S k • D : A shape estimate for each person p in our corpus • Q : A pose deformation model. For each scan S k the resulting model M( k , D , Q( k ) ) should closely match each alignment T k . Approach Coregistration approaches model learning and template registration simultaneously. The template is aligned to each scan while an articulated model is estimated to fit all alignments. SCAPE and BlendSCAPE Models or articulated 3D Shape SCAPE [2] and BlendSCAPE realistically deform a segmented template mesh T to have pose and shape D. Before each part of T is rotated by R(), D applies a per triangle shape deformation and Q() applies a per triangle pose correction. In SCAPE R() applies the same rigid rotation to every triangle within each part. In BlendSCAPE, triangles are rotated by weighted averages of the part rotations. BlendSCAPE smooths away artifacts, making Q() easier to learn. Model M Template T Scan S 1 Scan S 2 Alignment T 1 Alignment T 2 Results We coregister a corpus of several hundred 3D scans featuring multiple individuals in a wide range of poses. Coregistration improves the model’s ability to fit test and training data and yields visually accurate alignments. Scan S k Alignment T k Model M( k , D , Q( k )) Template T R rigid () R blend () R blend ()D R blend ()DQ() Scan SCAPE coreg after 4 iterations BlendSCAPE coreg after 4 iterations coreg after 1 iterations coreg after 1 iterations References [1] N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, H.-P. Seidel, A statistical model of human pose and body shape. Computer Graphics Forum. 28:2 (2009) 337–346 [2] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, J. Davis. SCAPE: Shape completion and animation of people. ACM ToG. 24 (2005) 408–416 Problem: Chicken-and-egg Model learning relies on scans that are accurately registered to a common 3D template. Accurate registration is difficult without a good model of how the template is allowed to deform. [1] Alignment-to-Scan Distance Alignment-to-Model Distance Model Regularization p k Distance to Scan iteration cm 0 1 2 3 4 5 6 2.0 1.5 1.0 0.5 0.0 p k p k p k

Upload: others

Post on 17-Feb-2022

2 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Coregistration: Simultaneous Alignment and Modeling of

The OptimizationBy minimizing a single objective function, we reliably obtain high quality alignments to noisy scans while simultaneously learning a realistic articulated body model.

Coregistration alternates between updating each alignment Tk and pose 𝜃k, each

person’s body shape D , and the global pose deformation model Q(𝜃).

Coregistration: Simultaneous Alignment and Modeling of Articulated 3D Shape

David A. Hirshberg Matthew Loper Eric Rachlin Michael J. BlackMPI for Intelligent Systems, Perceiving Systems Department

The Corpus Registration ProblemGoal: Align an entire corpus of body scans. Build a highly realistic 3D body model.

Solution: Coregistration simultaneously registers the corpus and builds the model.

ChallengeRegister a 3D template mesh to each scan. Each aligned template must closely match the surface of each scan. All alignments must also be in accurate point-to-point correspondence.

GoalAn accurate 3D model of the body that captures non-rigid pose deformation and individual shape variations across a population.

OutputsWhen coregistering a corpus of scans we obtain:

• Tk : An alignment to each scan Sk

• 𝜃k : A pose estimate for each scan Sk

• D : A shape estimate for each person p in our corpus• Q : A pose deformation model.

For each scan Sk the resulting model M( 𝜃k, D , Q(𝜃k) ) should closely match each alignment Tk.

ApproachCoregistration approaches model learning and template registration simultaneously. The template is aligned to each scan while an articulated model is estimated to fit all alignments.

SCAPE and BlendSCAPEModels or articulated 3D Shape

SCAPE [2] and BlendSCAPE realistically deform a segmented template mesh T to have pose 𝜃 and shape D.

Before each part of T is rotated by R(𝜃), D applies a per triangle shape

deformation and Q(𝜃) applies a per triangle pose correction.

In SCAPE R(𝜃) applies the same rigid rotation to every triangle within each part. In

BlendSCAPE, triangles are rotated by weighted averages of the part rotations.

BlendSCAPE smooths away artifacts, making Q(𝜃) easier to learn.

Model MTemplate TScan S1

Scan S2

Alignment T1

Alignment T2

ResultsWe coregister a corpus of several hundred 3D scans featuring multiple individuals in a wide range of poses. Coregistration improves the model’s ability to fit test and training data and yields visually accurate alignments.

ScanSk

AlignmentTk

Model M(𝜃k, D , Q(𝜃k))

Template T Rrigid(𝜃) Rblend(𝜃) Rblend(𝜃)D Rblend(𝜃)DQ(𝜃)

Scan SCAPE

coreg after 4 iterations

BlendSCAPE

coreg after 4 iterations

coreg after 1 iterations

References[1] N. Hasler, C. Stoll, M. Sunkel, B. Rosenhahn, H.-P. Seidel, A statistical model of human pose and body shape. Computer Graphics Forum. 28:2 (2009) 337–346

[2] D. Anguelov, P. Srinivasan, D. Koller, S. Thrun, J. Rodgers, J. Davis. SCAPE: Shape completion and animation of people. ACM ToG. 24 (2005) 408–416

Problem: Chicken-and-egg

Model learning relies on scans that are accurately registered to a common 3D template.

Accurate registration is difficult without a good model of how the template is allowed to deform. [1]

Alignment-to-Scan Distance

Alignment-to-Model Distance

Model Regularization

Distance to Scan

iteration

0 1 2 3 4 5 6

2.0

1.5

1.0