1. SIGGRAPH Asia 2011 Preview Seminar - Material Editing - Yoshihiro Kanamori 2011 Nov. 25

2. Material Editing Material Matting – Daniel Lepage, Jason Lawrence (University of Virginia) Physically-Based Interactive Bi-Scale Material Design – Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University) AppGen: Interactive Material Modeling from a Single Image – Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia), Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia) AppWarp: Retargeting Measured Materials by Appearance-space Warping – Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia), Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

3. Material Editing Material Matting – Daniel Lepage, Jason Lawrence (University of Virginia) Physically-Based Interactive Bi-Scale Material Design – Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University) AppGen: Interactive Material Modeling from a Single Image – Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia), Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia) AppWarp: Retargeting Measured Materials by Appearance-space Warping – Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia), Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

4. Material Matting Given Spatially-Varying BRDFs (SVBRDFs), separates them into foreground (F) / background (B) layers, and allows editing Input SVBRDF and separationEdited components and recomposited SVBRDF

5. Matting? Standard matting equation – Many methods exist for image matting This method peels apart SVBRDF V as layers L i one by one – Extracted regions are completed by texture synthesis

6. Overview Interactive system – Users draws scribbles to support separation, like recent image matting methods

7. Formulation Based on a Bayesian approach, i.e., Maximum A Posteriori (MAP) estimate Take –log( ), and for α minimize – Likewise, for F and B

9. Material Editing Material Matting – Daniel Lepage, Jason Lawrence (University of Virginia) Physically-Based Interactive Bi-Scale Material Design – Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University) AppGen: Interactive Material Modeling from a Single Image – Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia), Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia) AppWarp: Retargeting Measured Materials by Appearance-space Warping – Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia), Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

10. Physically-Based Interactive Bi-Scale Material Design Not available now (coming soon)

11. Material Editing Material Matting – Daniel Lepage, Jason Lawrence (University of Virginia) Physically-Based Interactive Bi-Scale Material Design – Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University) AppGen: Interactive Material Modeling from a Single Image – Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia), Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia) AppWarp: Retargeting Measured Materials by Appearance-space Warping – Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia), Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

12. AppGen: Interactive Material Modeling from a Single Image Extracts {specular/diffuse/roughness components, height field, and normal map} from a single image, for relighting applications Contribution: proposal of the whole system – Technically, diffuse extraction and normal map construction are improved

13. FYI: Bitmap2Material Similar commercial tool by allegorithmic

14. Overview 1.Highlight/shadow removal – Simple thresholding for image intensities 2.Diffuse shading separation – Just like Intrinsic Images [Bousseau et al. 2009] – but different formulation, better results 3.Normal reconstruction – Shape from shading→height field→normal estimate 4.Specular assignment – User assigns preset BRDFs (Ward model)

15. Comparisons Diffuse shading separation – Fewer user inputs than [Bousseau et al. 2009] Normal reconstruction – Better results than others

17. Failure Cases See Figure 13 W/ strong geometric structures W/ large regions of highlights Grayscale image, where the albedo only has grayscale variations fails to separate the black text from shading variations and generates artifacts in the normal map

18. Material Editing Material Matting – Daniel Lepage, Jason Lawrence (University of Virginia) Physically-Based Interactive Bi-Scale Material Design – Hongzhi Wu, Julie Dorsey, Holly Rushmeier (Yale University) AppGen: Interactive Material Modeling from a Single Image – Yue Dong (Tsinghua University), Xin Tong (Microsoft Research Asia), Fabio Pellacini (Dartmouth College / apienza University of Rome), Baining Guo (Microsoft Research Asia) AppWarp: Retargeting Measured Materials by Appearance-space Warping – Xiaobo An (Dartmouth College), Xin Tong (Microsoft Research Asia), Jonathan Denning, Fabio Pellacini (Dartmouth College / apienza University of Rome)

19. AppWarp: Retargeting Measured Materials by Appearance-space Warping Given measured materials (w/ SVBRDFs), the appearance of a source material is retargeted to that of a template material – Just like color transfer

20. Overview 1.Calculates the distance between each pair of BRDF samples [Pellacini and Lawrence 2007] 2.Embeds the samples into low-dimensional (two or three) appearance space using multidimensional scaling (MDS) 3.Warp the appearance space using Moving Least Squares (MLS) Warping [Schaefer et al. 2006] – Affine transformation only, w/ 2k-10k control points

21. Multidimensional Scaling (MDS)? Given pair-wise distances between samples (dissimilarity matrix) Calculate positions of N-dimensional samples {x i } so that the distances are preserved