1 technische universität münchen, 2 siemens corporate ......miroslava slavcheva1,2, maximilian...
Post on 23-Mar-2021
1 Views
Preview:
TRANSCRIPT
KillingFusion: Non-rigid 3D Reconstruction without Correspondences
Miroslava Slavcheva1,2, Maximilian Baust1, Daniel Cremers1, Slobodan Ilic1,2
1 Technische Universität München, 2 Siemens Corporate Technology
Task: real-time non-rigid 3D reconstruction
Multiple objectsInput: single, noisy
depth stream.
#0 #50 #85 #94
Warped
live frames.
Output: canonical-
pose model.
Quantitative evaluation dataset
Reconstruction
error [mm]
VolumeDeform
[Innmann16]
KillingFusion
[ours]
Duck 5.362 3.896
Snoopy 4.205 3.543
Kill
ing
Fusio
nV
olu
me
Defo
rm
Liv
e
fra
me
Topology changes & fast motion
Results
Objectives
• Topological changes. • Rapid inter-frame motion.
• Handle only contrived motion. [all]
• Processing takes hours. [Dou15]
• Intermittent conversion from SDF to mesh for cor-
respondence estimation. [Newcombe15, Innmann16]
• Need to estimate a 6D deformation field. [all]
Limitations of related approaches
References[Dou15] 3D Scanning Deformable Objects
with a Single RGBD Sensor, CVPR 2015.
[Newcombe15] DynamicFusion:
Reconstruction and Tracking of Non-rigid
Scenes in Real-Time, CVPR 2015.
[Innmann16] VolumeDeform: Real-time
Volumetric Non-rigid Reconstruction, ECCV
2016.
Contributions
• Full non-rigid framework based on a single data
representation – the signed distance field (SDF).
• Inherently handles topological changes.
• No correspondence estimation.
• No intermittent mesh conversion.
SDF mesh
Proposed approach
Incremental variational estimation of a 3D defor-
mation field Ψ = (u,v,w) from the projective SDF Φn of
the current frame towards canonical-pose model Φglobal.
• Data term: alignment.
• Level set regularizer: geometric correctness.
• Motion regularizer: rigidity.
KillingdampedK
setlevelSdata EEEE
2,,
,,,,2
1
zyx
globalndata zyxwzvyuxE
2,,
1,,2
1
zyx
nsetlevel wzvyuxE
JvecJvecJvecJvecETT
zyx
T
Killingdamped
,,
Scene flow:
Killing vector field
(divergence-free): JvecJvecJvecJvecJJ
TT
zyx
T
zyxF
T ,,,,
2
2
1
zyx zyx
TJvecJvecwvu
,, ,,
222
Other datasets
Effects of energy components
All terms wS = 0:
no level set
regularizer
wK = 0:
no motion
regularizer
γ = 0:
no Killing
component
γ = 1:
pure Killing
condition
Full 3D
Limitations
• No correspondences –
cannot propagate texture.
• Regular voxel grid – real-
time limited to ~803 voxels.
top related