siggraph asia 2011 preview seminar - material editing - yoshihiro kanamori 2011 nov. 25

SIGGRAPH Asia 2011 Preview Seminar- Material Editing -

Yoshihiro Kanamori2011 Nov. 25

Material Editing

• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)

• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)

• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),

Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)

• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),

Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

Material Editing

• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)

• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)

• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),

Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)

• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),

Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

Material Matting

• Given Spatially-Varying BRDFs (SVBRDFs), separates them into foreground (F) / background (B) layers, and allows editing

Input SVBRDF and separation Edited components and recomposited SVBRDF

Matting?

• Standard matting equation

– Many methods exist for image matting

• This method peels apart SVBRDF V as layers Li one by one

– Extracted regions are completed by texture synthesis

Overview

• Interactive system– Users draws scribbles to support separation,

like recent image matting methods

Formulation

• Based on a Bayesian approach, i.e., Maximum A Posteriori (MAP) estimate

• Take –log( ), and for α minimize

– Likewise, for F and B

Material Editing

• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)

• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)

• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),

Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)

• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),

Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

Physically-Based Interactive Bi-Scale Material Design

• Not available now (coming soon)

Material Editing

• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)

• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)

• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),

Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)

• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),

Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

AppGen: Interactive Material Modeling from a Single Image

• Extracts {specular/diffuse/roughness components, height field, and normal map} from a single image, for relighting applications

• Contribution: proposal of the whole system– Technically, diffuse extraction and normal map

construction are improved

FYI: Bitmap2Material

• Similar commercial tool by allegorithmic

Overview

1. Highlight/shadow removal– Simple thresholding for image intensities

2. Diffuse shading separation– Just like Intrinsic Images [Bousseau et al. 2009] – but different formulation, better results

3. Normal reconstruction– Shape from shading→height field→normal estimate

4. Specular assignment– User assigns preset BRDFs (Ward model)

Comparisons

• Diffuse shading separation– Fewer user inputs than

[Bousseau et al. 2009]

• Normal reconstruction– Better results

than others

Failure Cases

• See Figure 13

W/ strong geometric structures

W/ large regions of highlights

Grayscale image, where the albedo only has grayscale variations

fails to separate the black text from shading variations and generates artifacts in the normal map

Material Editing

• Material Matting– Daniel Lepage, Jason Lawrence (University of Virginia)

• Physically-Based Interactive Bi-Scale Material Design– Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University)

• AppGen: Interactive Material Modeling from a Single Image– Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia),

Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia)

• AppWarp: Retargeting Measured Materials by Appearance-space Warping– Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia),

Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

AppWarp: Retargeting Measured Materials by Appearance-space Warping

• Given measured materials (w/ SVBRDFs), the appearance of a source material is retargeted to that of a template material – Just like color transfer

Overview

1. Calculates the distance between each pair of BRDF samples [Pellacini and Lawrence 2007]

2. Embeds the samples into low-dimensional (two or three) appearance space using multidimensional scaling (MDS)

3. Warp the appearance space using Moving Least Squares (MLS) Warping [Schaefer et al. 2006]– Affine transformation only, w/ 2k-10k control points

Multidimensional Scaling (MDS)?

• Given pair-wise distances between samples(dissimilarity matrix)

• Calculate positions of N-dimensional samples {xi} so that the distances are preserved