2. Image-Guided Weathering: A New Approach Applied to Flow Phenomena C. Bosch 1, P. Y. Laffont, H. Rushmeier, J. Dorsey, G. Drettakis Yale University – REVES/INRIA Sophia Antipolis 1 Currently at ViRVIG, University of Girona

3. Aging and Weathering   Essential for modeling urban environments   Governed by physical, chemical and biological processes

4. Flow effects   Particularly complex   Flow over the scene (global effect)   Material properties (local effect)

5. Aging and Weathering in CG   Physically-based simulation   Difficult to get the desired effect   Texture synthesis   Restricted by input information   Global effects particularly hard

6. Motivation   Physically-based simulation   More flexible, allows global effects   Two main difficulties   Choosing appropriate parameters to achieve a given effect   Obtaining realistic visual detail

7. Image-Guided Weathering   Use images to guide simulation   Flow stains as a representative case Exemplar New simulation

8. Overview (I)   Extract data from exemplars   Color information   Simulation parameters   High frequency details S i = 1.301 r t = 0.252 k S = 0.0201 a t = 0.404 k D = 0.0807 T = 803 k a,t = 0.021 Exemplar Data

9. Overview (II)   Simulate new effects on scenes S i = 1.301 r t = 0.252 k S = 0.0201 a t = 0.404 k D = 0.0807 T = 803 k a,t = 0.021 Data

10. Related Work   Simulation   Phenomenon-specific [Merillou08]   Flow stains [Dorsey96; Chen05; Endo10]   Capture-and-transfer (synthesis)   Single image [Wang06; Xue08]   Acquisition systems [Gu06; Mertens06; Sun07; Lu07]   Inverse procedural textures [Bourque04; Lefebvre00]

11. Flow model   Particle-based simulation [Dorsey96]   Absorption, solubility and deposition   Stain concentration maps   Parameters   Particles: mass (m), Si   Stain material: kS, kD   Target materials: a, ka, roughness (r)   Simulation: time (t), particle rate (N)

12. Extracting Stains   Based on Appearance Manifolds [Wang06] Exemplar Appearance Manifold Degree Map

13.   Degree map = Stain concentration map ErrorSimulation Parameter Fitting Input stainDegree map S i = 1 k S = 0.04 k D = 0.04 r t = 0.2 a t = 0.3 k a,t = 0.05 T = 300 S i = 1 k S = 0.04 k D = 0.04 r t = 0.2 a t = 0.3 k a,t = 0.05 T = 300 Initial parameters New parameters S i = 1.3 k S = 0.02 k D = 0.08 r t = 0.25 a t = 0.4 k a,t = 0.02 T = 803 S i = 1.3 k S = 0.02 k D = 0.08 r t = 0.25 a t = 0.4 k a,t = 0.02 T = 803 Error < threshold or max. iterations Stop Proxy geometry target (Levenberg-Marquardt) [Lourakis04] image plane source

14. Improving Fitting 1. 1. Stain distribution along the source   Accumulate degree from bottom to top

15. Improving Fitting (II) 2. 2. Flow deflection along the target   Compute local degree distribution (~vector field)

16. ErrorSimulation Input stainDegree map S i = 1 k S = 0.04 k D = 0.04 r t = 0.2 a t = 0.3 k a,t = 0.05 T = 300 S i = 1 k S = 0.04 k D = 0.04 r t = 0.2 a t = 0.3 k a,t = 0.05 T = 300 Initial parameters New parameters S i = 1.3 k S = 0.02 k D = 0.08 r t = 0.25 a t = 0.4 k a,t = 0.02 T = 803 S i = 1.3 k S = 0.02 k D = 0.08 r t = 0.25 a t = 0.4 k a,t = 0.02 T = 803 Error < threshold or max. iterations Stop Proxy geometry target (Levenberg-Marquardt) [Lourakis04] image plane source Parameter Fitting (II) Vector field Stain distribution

17. Fitting Results (w/o vector field) ExemplarDegree MapSimulation Using source distribution

18. Fitting Results (w/o vector field) ExemplarDegree MapSimulation

19. Fitting Results (w/ vector field) Exemplar Degree MapSimulation w/o vfield

20. Fitting Results (w/ vector field) ExemplarDegree MapSimulation

21. Fitting Results (w/ vector field) ExemplarDegree MapSimulation

22. Fitting Results (Complex Targets) Exemplar Degree MapSimulation

23. Stain Detail   Simulation lacks spatial variations (high-frequency detail) Degree MapSimulation Exemplar

24. Detail Maps   Extract detail by image difference   Use guided texture synthesis [Lefebvre05]   Detail maps will modify stain adhesion Degree MapSimulationDifference Detail Map

25. Simulating New Stains   Link data to stain sources and targets   Parameters, detail maps, color   Use 1D texture synthesis for distributions   Run flow simulation   Flow deflected by target geometry (+ disp. map)

26. Color Transfer   Transfer stain color from input image   Background mixed with stain everywhere   Non-linear relationship between color and degree   Use per-pixel warping background color target background fully stained

27. Results

28. Results (II)

29. Results (III)

30. Results (IV)

31. Performance   Preprocessing   Degree map: 1-3 minutes   Fitting: 30-60 minutes (500 iter., ~256x512)   Detail synthesis: 1-2 minutes (1024x1024)   Final simulation   Stain simulation: 2-5 minutes/stain   Color warping: 5-8 seconds/stain (1024x1024)

32. Limitations   Good extraction from background   Fitting: Not true physical estimations   Detail maps: Depend on appropriate fit   Computation time

33. Conclusions   New approach to acquire simulation data from photographs   Solves parameter estimation from images   Combines simulation with data-driven methods   Appearance manifold, texture synthesis, …   Fills the gap between data-driven and simulation   Easy to use   Natural variations (including global effects)

34. Future work   Extend to other weathering phenomena   Deal with large scale scenes   Fast simulation, global effects, …

35. Acknowledgements   Visiting grant U.Girona   ANR project (ANR-06-MDCA-004-01)   ERCIM “Alain Bensoussan” Fellowship   Autodesk (Maya/MentalRay)   Coding help: Li-Ying, Su Xue   Scene treatment: S. Close and F. Andrade-Cabral

36. Thank you