faces: analysis and synthesis
DESCRIPTION
Faces: Analysis and Synthesis. Vision for Graphics CSE 590SS, Winter 2001 Richard Szeliski. What can we do with faces?. Modeling (reconstruction): manual [Pighin et al. 1998] Automated [Zhang et al. 2000]. What can we do with faces?. Analysis - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/1.jpg)
Faces: Analysis and Synthesis
Vision for GraphicsCSE 590SS, Winter 2001
Richard Szeliski
![Page 2: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/2.jpg)
2/14/2001 Vision for Graphics 2
What can we do with faces?
Modeling (reconstruction):• manual
[Pighin et al. 1998]
• Automated[Zhang et al. 2000]
![Page 3: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/3.jpg)
2/14/2001 Vision for Graphics 3
What can we do with faces?
Analysis• principal components and deformation modes
[Turk & Pentland 1991][Rowland & Perrett 1995][Guenter et al. 1998][Blanz & Vetter 1999]
![Page 4: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/4.jpg)
2/14/2001 Vision for Graphics 4
What can we do with faces?
Tracking and synthesis• tracking
[Toyama 1998]• animation
[Pighin et al. 1999][Buck et al. 2000]
Recognition[Turk & Pentland 1991; Lanitis et al. 1997]
![Page 5: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/5.jpg)
2/14/2001 Vision for Graphics 5
Synthesizing Realistic Facial Expressions from Photographs
Frederic Pighin Jamie Hecker Dani Lischinski David Salesin Richard Szeliski *
SIGGRAPH’98
![Page 6: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/6.jpg)
Animated Face Modeling From Video Images
Zhengyou Zhang, Zicheng Liu, Michael Cohen
Vision Group & Graphics GroupMicrosoft Research
![Page 7: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/7.jpg)
Manipulating Facial Appearance through Shape and Color
Duncan A. Rowland and David I. PerrettSt Andrews University
IEEE CG&A, September 1995
![Page 8: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/8.jpg)
2/14/2001 Vision for Graphics 8
Principal component analysis
Compute average faces (color and shape)
Compute deviations between male and female (vector and color differences)
![Page 9: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/9.jpg)
2/14/2001 Vision for Graphics 9
Changing gender
Deform shape and/or color of an input face in the direction of “more female”
original shape
color both
![Page 10: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/10.jpg)
2/14/2001 Vision for Graphics 10
Enhancing gender
more same original androgynous more opposite
![Page 11: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/11.jpg)
2/14/2001 Vision for Graphics 11
Changing age
Face becomes “rounder” and “more textured” and “grayer”
original shape
color both
![Page 12: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/12.jpg)
A Morphable Model For The Synthesis Of 3D Faces
Volker Blanz Thomas VetterSIGGRAPH’99
![Page 13: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/13.jpg)
2/14/2001 Vision for Graphics 13
Morphable model of 3D faces
Start with a catalogue of 200 3D Cyberware scans
Build a model of average shape and texture, and principal variations
![Page 14: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/14.jpg)
2/14/2001 Vision for Graphics 14
Morphable model of 3D facesDivide face into 4 regions (eyes, nose, mouth, head)
For each new prototype, find amount of deviation from the reference shape and texture.
![Page 15: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/15.jpg)
2/14/2001 Vision for Graphics 15
Morphable model of 3D faces
Adding some variations
![Page 16: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/16.jpg)
2/14/2001 Vision for Graphics 16
Reconstruction from single image
![Page 17: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/17.jpg)
2/14/2001 Vision for Graphics 17
Modifying a single image
![Page 18: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/18.jpg)
2/14/2001 Vision for Graphics 18
Animating from a single image
![Page 19: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/19.jpg)
2/14/2001 Vision for Graphics 19
Resulting animation
![Page 20: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/20.jpg)
Tracking Face Orientation
Kentaro ToyamaVision–Based Interaction Group
Microsoft Research
![Page 21: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/21.jpg)
2/14/2001 Vision for Graphics 21
1University of Washington 2Microsoft Research
ICCV’99
Resynthesizing Facial Animation through 3D Model-Based Tracking
Frédéric Pighin1 Richard Szeliski2 David Salesin1,2
![Page 22: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/22.jpg)
Performance-Driven Hand-Drawn Animation
Ian Buck Adam Finkelstein Charles Jacobs Allison Klein
David H. Salesin Joshua Seims Richard Szeliski Kentaro Toyama
![Page 23: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/23.jpg)
2/14/2001 Vision for Graphics 23
Go from video to “cartoon”
![Page 24: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/24.jpg)
2/14/2001 Vision for Graphics 24
Hand-drawn expressions
![Page 25: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/25.jpg)
2/14/2001 Vision for Graphics 25
Expression correspondence
![Page 26: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/26.jpg)
2/14/2001 Vision for Graphics 26
Morphing and tracking
![Page 27: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/27.jpg)
2/14/2001 Vision for Graphics 27
More examples
![Page 28: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/28.jpg)
2/14/2001 Vision for Graphics 28
Final animations
![Page 29: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/29.jpg)
2/14/2001 Vision for Graphics 29
BibliographyF. Pighin, J. Hecker, D. Lischinski, D. H. Salesin, and R. Szeliski.
Synthesizing realistic facial expressions from photographs. In SIGGRAPH'98 Proceedings, pages 75--84, Orlando, July 1998.
Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen. Rapid modeling of animated faces from video. Technical Report MSR-TR-2000-11, Microsoft Research, February 2000.
B. Guenter et al. Making faces. Proceedings of SIGGRAPH 98, pages 55--66, July 1998.
V. Blanz and T. Vetter. A morphable model for the synthesis of 3d faces. Proceedings of SIGGRAPH 99, pages 187--194, August 1999.
K. Toyama. Prolegomena for robust face tracking. Technical Report MSR-TR-98-65, Microsoft Research, November 1998.
F. Pighin, D. H. Salesin, and R. Szeliski. Resynthesizing facial animation through 3D model-based tracking. In Seventh International Conference on Computer Vision (ICCV'99), pages 143--150, Kerkyra, Greece, September 1999.
![Page 30: Faces: Analysis and Synthesis](https://reader035.vdocuments.us/reader035/viewer/2022062814/568167d5550346895ddd2d55/html5/thumbnails/30.jpg)
2/14/2001 Vision for Graphics 30
BibliographyI. Buck et al. Performance-driven hand-drawn animation. In Symposium on
Non Photorealistic Animation and Rendering, pages 101--108, Annecy, June 2000. ACM SIGGRAPH.
D. A. Rowland and D. I. Perrett. Manipulating facial appearance through shape and color. IEEE Computer Graphics and Applications, 15(5):70--76, September 1995.
M. Turk and A. Pentland. Face recognition using eigenfaces. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'91), pages 586--591, Maui, Hawaii, June 1991. IEEE Computer Society Press.
P. N. Belhumeur, J. P. Hespanha, and D. J. Kriegman. Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):711--720, July 1997.
A. Lanitis, C. J. Taylor, and T. F. Cootes. Automatic interpretation and coding of face images using flexible models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7):742--756, July 1997.