towards accurately extracting facial expression parameters
DESCRIPTION
Towards Accurately Extracting Facial Expression Parameters. Xuecheng Liu Dinghuang Ji Zhaoqi Wang and Shihong Xia { liuxuecheng,jidinghuang,zqwang,xsh }@ ict.ac.cn Institute of Computing Technology, Chinese Academy of Sciences, China. Outline. Background Problem Our Method - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/1.jpg)
Towards Accurately Extracting Facial Expression Parameters
Xuecheng Liu Dinghuang Ji Zhaoqi Wang and Shihong Xia{liuxuecheng,jidinghuang,zqwang,xsh}@ict.ac.cn
Institute of Computing Technology, Chinese Academy of Sciences, China
![Page 2: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/2.jpg)
Outline
Background Problem Our Method Experiments Summary
![Page 3: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/3.jpg)
Background
Applications of facial animation
Movies Games Tele-Immersion
![Page 4: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/4.jpg)
Background
• Blendshapes
1
0.8w
2
0.5w
...
![Page 5: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/5.jpg)
Background
Vicon Captured Data
Head Motion
Rotation Transition
Facial Expression
Blendshapes Weights
![Page 6: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/6.jpg)
ProblemUsually,
• Head absolute orientation is first extracted, including head rotation matrix R and transition vector t.
• The captured facial expression data are then transformed to local face coordinates system.
• However, the assumption that four markers are relative fixed may not always be true. This results to error of computing head orientation. Then, above error will be amplified during computing expressions parameters w.
![Page 7: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/7.jpg)
Problem
• [HAV 06] propose a hierarchical manner– Doing a global or gross stabilization choosing
selective markers (head, ears and the nose bone)– Refining the output by local or fine stabilization by
analyzing the marker movements relative to a facial surface model
![Page 8: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/8.jpg)
Our Method
• To undermine the error accumulation, we propose a revision based optimization method.
![Page 9: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/9.jpg)
Our Method
• To undermine the error agglomeration, we propose a revision based optimization method.Simplify
![Page 10: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/10.jpg)
Our Method
• To undermine the error agglomeration, we propose a revision based optimization method.
• Then we use a traditional alternative method and non-negative least square to solve the problem.
![Page 11: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/11.jpg)
Our Method
• Linear blendshape
• Nonlinear blendshape
![Page 12: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/12.jpg)
Experiments
• Data and Environment– Training data: 500 frames– Testing data: 15 segments( #Frame
>36,000)– Ratio >= 80– Basic blendshapes: 27个– Hardware : x86 2.67G Core2 CPU, 2G RAM– Software:Matlab
![Page 13: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/13.jpg)
Experiments
• We compare with two published which are both representative works in areas.– [HAV 06] is a typical linear blendshape based
method, this work is presented as course of Siggraph 2006 and used in many popular movies.
– [LXFW 11] is a typical nonlinear blendshape based method, and is published in CGF 2011.
![Page 14: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/14.jpg)
Experiments
• Contrast experiment 1 (with [HAVALDAR 06])[HAVALDAR 06] [Our Method]
![Page 15: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/15.jpg)
Experiments
• Contrast experiment 1 (with [HAV 06])
0%2%4%6%8%
10%12%14%16%18%20%22%
Faci al ani mat i on cl i p
Rela
tive
Err
or
[Hav06] Our Method
0. 00. 20. 40. 60. 81. 01. 21. 4
Faci al ani mat i on cl i p
Abso
lute
Err
or(m
m)
[Hav06] Our Method
Absolute Error
Relative Error
(b)(a)
![Page 16: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/16.jpg)
Experiments
• Contrast experiment 1 (with [HAV 06])
0
5
10
15
Faci al ani mat i on cl i p
Aver
age
Iter
atio
ns
[Hav06] Our Method
020
4060
80100
120
Faci al ani mat i on cl i p
Fram
e ra
tes
f/s
()
[Hav06] Our Method
(b)(a)
![Page 17: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/17.jpg)
Experiments
![Page 18: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/18.jpg)
Experiments
• Contrast experiment 2 (with [LXFW11])[LXFW11] Our Method]
![Page 19: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/19.jpg)
Experiments
• Contrast experiment 2 (with [LXFW11])Absolute Error
Relative Error
(b)(a)
0. 0
0. 5
1. 0
1. 5
2. 0
2. 5
Faci al ani mat i on cl i p
Abso
lute
err
ormm
()
[LXFW11] Our Method
0%
5%
10%
15%
20%
25%
30%
35%
Faci al ani mat i on cl i p
Rela
tive
err
or
[LXFW11] Our Method
![Page 20: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/20.jpg)
Experiments
• Contrast experiment 2 (with [LXFW11])
(b)(a)
0
5
10
15
20
Faci al ani mat i on cl i p
Aver
age
iter
atio
ns
[LXFW11] Our Method
0. 0
10. 0
20. 0
30. 0
40. 0
50. 0
Faci al ani mat i on cl i pFr
ame
rate
sf/
s(
)
[LXFW11] Our Method
![Page 21: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/21.jpg)
Experiments
![Page 22: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/22.jpg)
Conclusion
Application Methods Precision promotion
Efficiency promotion
Linear blendshape
Our Method VS. [Hav06]
+12.3% +86.0%
Nonlinear blendshape
Our Method VS. [LXFW11]
+23.5% -18.2%
![Page 23: Towards Accurately Extracting Facial Expression Parameters](https://reader035.vdocuments.us/reader035/viewer/2022062517/56813711550346895d9e995d/html5/thumbnails/23.jpg)
Bibliography
• [HAV 06] HAVALDAR P. Sony pictures imageworks. In SIGGRAPH ’06: ACM SIGGRAPH 2006 Courses (New York, NY, USA, 2006), ACM, p. 5.
• [LXFW11] Xuecheng Liu, Shihong Xia, Shihong Xia, Yiwen Fan, Zhaoqi Wang. Exploring nonlinear relationship of blendshape facial animation. To appear in the Journal of Computer Graphics Forum.