Presentation is loading. Please wait.

Presentation is loading. Please wait.

Inverse Texture Synthesis

Similar presentations

Presentation on theme: "Inverse Texture Synthesis"— Presentation transcript:


2 Inverse Texture Synthesis
Li-Yi Wei1 Jianwei Han2 Kun Zhou1,2 Hujun Bao2 Baining Guo1 Harry Shum1 1Microsoft 2Zhejiang University

3 Example-based texture synthesis
For a small input texture produce an arbitrarily large output with similar look Why? may not possible to obtain large input texture synthesis input output

4 Inverse texture synthesis
From a large input texture produce a small output that best summarizes input inverse texture synthesis output input

5 Why? Textures are getting large Advances in scanning technology
High dimensionality: time-varying, BRDF Expensive to store, transmit, compute So why we need this? Textures are getting larger, making them more expensive to store, transmit, and compute. Yale University MSR Asia Columbia University

6 Overview inverse texture synthesis input output (large) (small)
texturing (slow) (fast) Inverse texture synthesis can solve these problems. Given a large input, the algorithm will produce a small compaction that retains vital information of the input. The compaction can be used for texturing with similar visual results from the original input, with faster computation and smaller memory consumption. similar quality

7 Related work: image compression
pixel-wise identical compress decompress inverse synth texture synth input perceptual similar

8 Related work: epitome Epitome [Jojic et al. 2003]
Jigsaw [Kannan et al. 2007] Major source of inspiration for us For general images, not just textures We provide better quality Bidirectional similarity [Simakov et al. 2008] Factoring repeated content [Wang et al. 2008]

9 Related work: manual crop
stationary globally varying original manual crop our result

10 Globally-varying textures
Markov Random Field (MRF) textures local & stationary Globally-varying textures local, but not necessarily stationary MRF globally varying

11 Globally varying textures Previous work
MRF input → globally varying output texture-by-numbers in Image analogies [Hertzmann et al. 2001] progressively variant textures [Zhang et al. 2003] texture design and morphing [Matusik et al. 2005] Globally varying input appearance manifold [Wang et al. 2006] spatially & time varying BRDF [Gu et al. 2006] context-aware texture [Lu et al. 2007]

12 Globally varying textures Definition
texture + control maps Examples of control maps user-specified colors [Hertzmann et al. 2001] spatially-varying parameters [Gu et al. 2006] weathering degree-map [Wang et al. 2006] context information [Lu et al. 2007] texture (paint crack) control map (paint thickness)

13 Globally varying textures
Including time-varying textures as well Large data size! time-varying BRDF [Gu et al. 2006] 512 x 512 x 33, 288 MB context-aware texture [Lu et al. 2007] 1226 x 978 x 50, 35 MB

14 Inverse texture synthesis
Compacting globally varying textures including both texture + control map output compaction inverse synthesis texture control texture control map input

15 Compaction as summary of original
Re-synthesis with user control map faster slower forward synthesis user control + compaction re-synthesis from compaction re-synthesis from original

16 Inverse Texture Synthesis
Applicable to MRF textures no control map homogenized result manual cropping ? original re-synthesis

17 Inverse Texture Synthesis
Manual cropping unsuitable for globally variant texture no matter where you put the window our compaction manual crop black purple orange original from compaction from manual crop

18 forward term [Kwatra et al. 2005]
Basic formulation Inspired by texture optimization [Kwatra et al. 2005] inverse term (New!) forward term [Kwatra et al. 2005] xp Zp best match zq xq best match Z (output) X (input)

19 Energy plot energy original compaction size

20 Why both terms? inverse term preserves all input features
forward inverse term preserves all input features forward term avoids artifacts in compaction f-only missing feature both i-only garbage both i-only discontinuity both

21 Comparing with epitome [Jojic et al. 2003]
Similar to our method but only inverse term blur, discontinuity epitome epitome our our original original

22 Comparing with epitome [Jojic et al. 2003] Re-synthesis
our epitome our original original

23 Solver How to solve this? Texture optimization [Kwatra et al. 2005]
Discrete solver [Han et al. 2006]

24 Optimization [Kwatra et al. 2005]
NO inverse term forward term [Kwatra et al. 2005] E-step fix xq argminz E(x,z) least square M-step fix Z argminxq |xq-zq|2 search fix xq xq Zq zq xq Z argminxq |xq-zq|2 X

25 forward term [Kwatra et al. 2005]
Our solver inverse term forward term [Kwatra et al. 2005] E-step fix xq argminz E(x,z) least square M-step (forward) fix Z argminxq |xq-zq|2 search xp xp zp xq Zq discrete solver [Han et al. 2006] M-step (inverse) fix xp argminzp |xp-zp|2 discrete solver zq xq Z argminxq |xq-zq|2 discrete solver X

26 Results

27 Results output control compaction 1282 original (paint crack)
799 x 546 from orig. 84555 sec from comp. 1131 sec

28 Compaction size on quality
799 x 546 5122 2562 1282

29 Compaction size on quality
799 x 546 (original) 5122 2562 1282

30 Re-synthesis without control map
stationary only comp. original re-synthesis

31 GPU synthesis – small texture better Extension from [Lefebvre & Hoppe 2005]
3 fps, original 6 fps, compact cheese mold 1214 x 1212 compaction 1282 3.5 fps, original 7.0 fps, compact dirt 271x481 original

32 Limitation: Correlation between texture & control
original reconstruction compaction

33 Why little squares. Instead of, e. g
Why little squares? Instead of, e.g. a set of texton [Leung & Malik 2001] Most general compaction utilizable for any synthesis algorithm GPU synthesis Looks nice I like little squares visual summary & visualization

34 Orientation field for anisotropic textures
Orientation field w as part of energy function E(x, z) → E(x, z; w) Good orientation field yields better solution comp. no w comp. with w original orientation field

35 Results: orientation field just mention this in a quick pass
Paris et al. 04 original manual + interp ours

36 Results: orientation field
Paris et al. 04 original manual + interp ours

37 Results: orientation field
Paris et al. 04 original manual + interp ours

38 Results: orientation field
Paris et al. 04 original manual + interp ours

39 Future work Higher dimensional textures
e.g. video General images, not just textures Bidirectional similarity [Simakov et al. CVPR 2008] Image compression

40 Acknowledgements Yale graphics group Columbia graphics group Sylvain Lefebvre Hughes Hoppe Matusik et al Jiaping Wang Xin Tong Jian Sun Frank Yu Bennett Wilburn Eric Stollnitz Dwight Daniels Reviewers Dinesh Manocha Ming Lin Chas Boyd Brandon Lloyd Avneesh Sud Billy Chen

41 Thank You!

Download ppt "Inverse Texture Synthesis"

Similar presentations

Ads by Google