avatar digitization from a single image for real-time ...ƒ¡力文.pdf · introduction input...
TRANSCRIPT
AVATAR DIGITIZATION FROM A SINGLE IMAGE FOR REAL-TIME RENDERING
Liwen Hu1,2* Shunsuke Saito1,2* Lingyu Wei1,2*
Koki Nagano1 Jeawoo Seo1 Jens Fursund1 Iman Sadeghi
Carrie Sun1 Yen-Chun Chen1 Hao Li1,2,3
Cosimo
Introduction
XboxOne Sims4
FinalFantasy14 NBA2K7
Introduction
Oculus
Introduction
inputsingleimage
Introduction
digitalavatarinputsingleimage
Related Works - Facial Modeling and Capture
[Caoetal.2016]
[Beeleretal.2010]
[Bradleyetal.2010]
Related Works - Facial Modeling and Capture
[Thiesetal.2016]
[Caoetal.2016][Saitoetal.2017]
[Beeleretal.2010]
[Bradleyetal.2010]
[Caoetal.2015]
Related Works - Hair Modeling and Capture
[Huetal.2014]
[Luoetal.2013]
[Parisetal.2008]
Related Works - Hair Modeling and Capture
[Chaietal.2016]
[Zhangetal.2017]
[Huetal.2015]
[Huetal.2014]
[Luoetal.2013]
[Parisetal.2008]
Related Works - Hair Modeling and Capture
Introduction
100x playback speed
Introduction
Overview
inputimage
Overview
inputimage
facesandhairsegmentaHon
Overview
inputimage
facesandhairsegmentaHon
facemodeling
Overview
inputimage
facesandhairsegmentaHon
facemodeling facetexturereconstrucHon
facialrigging(blendshapes,joint-based,secondarycomponents)
Overview
inputimage
facesandhairsegmentaHon
facemodeling
hairstyledigiHzaHon
facetexturereconstrucHon
facialrigging(blendshapes,joint-based,secondarycomponents)
Overview
inputimage
facesandhairsegmentaHon
facemodeling
hairstyledigiHzaHon
facetexturereconstrucHon
hairappearancematching(shader,texture,alphamask,bumpmap,color)
facialrigging(blendshapes,joint-based,secondarycomponents)
Overview
inputimage
facesandhairsegmentaHon
facemodeling
hairstyledigiHzaHon
facetexturereconstrucHon
hairappearancematching(shader,texture,alphamask,bumpmap,color)
facialrigging(blendshapes,joint-based,secondarycomponents)
real-Hme3Davatarrendering
Image Pre-Processing
inputimage faceregion hairregion
Image Pre-Processing
Face Digitization
Analysis-by-synthesis3Dheadmodeling[BlanzandVetter1999;Thiesetal.2016]
3D Face Digitization
Morphable Model [Blanz and Vetter 1998]
Appearance Modeling Pipeline
partialtexture PCAtexture
completephotorealistictexture
high-qualitytexturedatabase
Visibility Constraints
inputimage
w/ovisibilityconstraints
w/visibilityconstraints
Hair Digitization
Hair Digitization
strands-level strips-level
Database Generation
USC-HairSalon(343) ourheadmodel
ourheadmodel
+
artistcreated(89)USC-HairSalon(343)
Database Generation
+
hairstyleA hairstyleB
Database Generation
+ =
hairstyleA hairstyleB combination
Database Generation
strands-level
Database Generation
[ZhuandBridson2005]strands-level
Database Generation
[ZhuandBridson2005] [Luoetal.2013]strands-level
Database Generation
Hair Attribute Extration
hairlength
bald shaved short long
fringe
nofringe fullfringe rightfringe leftfringe
…
inputimage
Hair Attribute Extration
Shortcurly not spiky...hairattribute
extraction
inputimage
Hair Attribute Extration
Shortcurly not spiky...hairattribute
extraction
hairstyledatabase
reduceddataset
haircategorymatching
inputimage
Hair Attribute Extration
Shortcurly not spiky...hairattribute
extraction
hairstyledatabase
reduceddataset
haircategorymatching
segmentationandorientation
Hairstyle Retrieval
reduceddataset
segmentationandorientation
thumbnails
Hairstyle Retrieval
reduceddataset
segmentationandorientation
hairstyleretrieval
closesthairstyle
thumbnails
Hairstyle Retrieval
inputimage closesthairstyle
Hair Mesh Fitting
inputimage headfittingdeformation
Hair Mesh Fitting
inputimage maskdeformation
Hair Mesh Fitting
inputimage maskdeformation
Hair Mesh Fitting
inputimage collisiondetection
Hair Mesh Fitting
inputimage collisiondetection
Hair Mesh Fitting
haircurve
straight wavy curly dreadlock
Hair Rendering and Texturing
straight dreadlock wavy
haircurve
straight wavy curly dreadlock
Hair Rendering and Texturing
Hair Rendering and Texturing
inputhairmodel
Polystrip Patching Optimization
inputhairmodelmulti-viewscalpvisibilitymap
Polystrip Patching Optimization
inputhairmodelmulti-viewscalpvisibilitymap
iteration1
Polystrip Patching Optimization
inputhairmodelmulti-viewscalpvisibilitymap
iteration1 finalresult
Polystrip Patching Optimization
Results
Results
Results
Comparisons
Comparisons
Comparisons
Conclusion
Future Work
Future Work
Thanks!