structured face hallucination · 2013. 7. 5. · title: structured face hallucination author:...
TRANSCRIPT
Structured Face HallucinationChih-Yuan Yang, Sifei Liu, Ming-Hsuan Yang
Electrical Engineering and Computer Science, University of California, [email protected], [email protected], [email protected]
code available http://eng.ucmerced.edu/people/cyang35
Challenges
•How to handle various poses and expressions?•How to retrieve effective exemplars?•How to preserve the visual consistency in outputs?•How to reconstruct sharp contours?
Main Idea•Split a face image into three image structures: facialcomponents, contours, and smooth regions
•Label exemplar images in terms of pose and glasses•Exploit landmarks to handle component variety andtransfer high-frequency details of a whole exemplarcomponent
•Reconstruct contours with high-quality directionsand sharpness through novel priors
•Transfer details of smooth regions from exemplarpatches
Visual Consistency
•Gradients of a whole region → consistency of awhole region
•Direction-preserving upsampling → consistent edgedirection
•Sharpness priors → consistent and visually correctedge contrast
Algorithm
Integrate gradient maps U = wcUc + (1−wc)(weUe + (1−we)Ub
)Generate an output image Ih = argmin
I‖∇I −U‖2 s.t. (I ⊗G) ↓= Il
Facial components
Contours
Directional similarity fk(p) = exp(−‖P −Qk‖/σ), k = 1, . . . ,KDirection-preserving HR image Id = argmin
I
∑k ‖fk(I)−Tk‖
2 s.t. (I ⊗G) ↓= Il
Restore edge sharpness Ue(p) =m̄′pmp·Ud(p)
Smooth regions
Patch search accelerated by the PatchMatch method
Experimental Results
(a) Input (b) Irani91 (c) Yang10 (d) Ma10 (e) Liu07 (f) Proposed (g) Original
Conclusions•Contributions
I Split-and-merge approach to reconstruct details of a face imageI Exploit landmarks for effective exemplar searchI Novel statistical priors and upsampling method for simultaneously preserving edge
direction and sharpness•Advantages
I Effective and consistent high-frequency detailsI Robustness for various poses and expressions
Chih-Yuan Yang, Sifei Liu, Ming-Hsuan Yang (EECS, University of California, Merced) 26th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2013)