1. Kartic Subr Cyril Soler Frédo Durand Edge-preserving Multiscale Image Decomposition based on Local Extrema INRIA, Grenoble Universities MIT CSAIL

2. Multiscale image decomposition + + Medium Pixels Intensity Input Fine Coarse 1D

3. Motivation Detail enhancement Separating fine texture from coarse shading

4. What is detail?

5. Some examples

6. Related work Linear multiscale methods Edge-preserving approaches 1D Signal analysis

7. Related work : Linear multiscale methods Edge-preserving approaches 1D Signal analysis [Burt and Adelson 93] [Rahman and Woodell 97] [Pattanaik et al 98] [Lindeberg 94] Edges not preserved (Causes halos while editing)

8. Related work : Edge-preserving methods 1D Signal analysis [Farbman et al 08] [Fattal et al 07] [Bae et al 07] [Chen et al 07] Edge-aware Assume detail is low contrast

9. Related work: Empirical mode decomposition Linear multiscaleEdge-preserving approaches [Huang et al 98] Developed for 1D signals Detail depends on spatial scale Not edge-aware

10. Related work Edge-preserving approaches 1D Signal analysis Linear multiscale methods

11. Existing edge-preserving image decompositions Input Edge-preserving smoothing (e.g. bilateral filter) Base layer Detail layer (Input – Base) + Iteratively smooth input Recursively smooth base layer OR

12. Input Base layer Detail layer (Input – Base) + Edge-preserving smoothing (e.g. bilateral filter) Edge (preserved) Detail (smoothed) Existing edge-preserving image decompositions Assume detail is low-intensity variation

13. Challenge: Smoothing high-contrast detail Input

14. Challenge: Smoothing high-contrast detail Edge Low-contrast detail High-contrast detail

15. Conservative smoothing (bilateral filter with narrow range-Gaussian) Challenge: Smoothing high-contrast detail Edge preserved? Low-contrast detail smoothed? High-contrast detail smoothed?

16. Challenge: Smoothing high-contrast detail Edge preserved? Low-contrast detail smoothed? High-contrast detail smoothed? Aggressive smoothing (bilateral filter with wide range-Gaussian)

17. Example: Smoothing high-contrast detail Input[Farbman et al 2008] λ= 13, α = 0.2 [Farbman et al 2008] λ= 13, α = 1.2 Detail not smoothed Coarse features smoothed Edge smoothed

18. Our approach: Use local extrema Input Local maxima Local minima Detail = oscillations between local extrema

19. Our approach: Use local extrema Base = Local mean of neighboring extrema

20. Our approach: Use local extrema Local mean of neighboring extrema Edge preserved? Low-contrast detail smoothed? High-contrast detail smoothed?

21. Our detail extraction Input Base layer Detail layer + High-contrast detail smoothed Edges preserved

22. Algorithm Identify local extrema Estimate smoothed mean Detail at multiple scales Input: Image + number of layers

23. Algorithm: Illustrative example

24. Algorithm: Identifying local extrema Extrema detection kernel Local maxima Local minima

25. Algorithm: Estimating smoothed mean 1) Construct envelopes Minimal envelope Interpolation preserves edge [Levin et al 04] Maximal envelope

26. Algorithm: Estimating smoothed mean 2) Average envelopes Estimated mean

27. Algorithm: After one iteration + Input Base Detail

28. Algorithm: Mean at coarser scale Local maxima Local minima Widen extrema detection kernel

29. Algorithm: Mean at coarser scale Minimal envelope Maximal envelope

30. Algorithm: Mean at coarser scale Estimated mean

31. Recap: Detail extraction Identify local extrema Construct envelopes Average envelopes - Input Detail Smoothed mean (Base)

32. Identify local extrema Construct envelopes Average envelopes Recap: Detail extraction Smoothed mean Detail = Input - BaseBase Input

33. Base B 2 Base B 1 Input Detail D 2 Detail D 1 Recap: Multiscale decomposition Layer 1Layer 2Layer 3 Iteration 1 on input Iteration 2 on B 1 Recurse n-1 times for n-layers CoarseFine

34. Results

35. Results: Smoothing Input Smoothed

36. Results: Multiscale decomposition Medium Input Fine Coarse Low contrast edgeHigh contrast detailLow contrast edgeHigh contrast detail

37. Results: Multiscale decomposition Input

38. Results: Multiscale decomposition Fine Coarse

39. Results: Multiscale decomposition Input

40. Results: Multiscale decomposition After one iteration Base layerDetail layer

41. Results: Multiscale decomposition After two iterations Base layerDetail layer

42. Applications: Image equalization

43. Applications: Smoothing hatched images

44. Applications: Coarse illumination transfer

47. Applications: Tone-mapping HDR images

48. Comparison [Farbman et al 2008] Our Result

49. Our smoothing

50. Limitation Input Our Result

51. Conclusion Detail based on local extrema Smoothing high contrast detail Edge-preserving multiscale decomposition

52. Acknowledgements INRIA post-doctoral fellowship Equipe Associée with MIT ‘Flexible Rendering’ Adrien Bousseau & Alexandrina Orzan HFIBMR grant (ANR-07-BLAN-0331) Anonymous reviewers

53. C++ source: http://artis.imag.fr/~Kartic.Subr/research.html