1. Applied Perception in Graphics Erik Reinhard University of Central Florida reinhard@cs.ucf.edu

2. Computer Graphics Produce computer generated imagery that cannot be distinguished from real scenesProduce computer generated imagery that cannot be distinguished from real scenes Do this in real-timeDo this in real-time

3. Trends in Computer Graphics Greater realismGreater realism –Scene complexity –Lighting simulations Faster renderingFaster rendering –Faster hardware –Better algorithms Together: still too slow and unrealisticTogether: still too slow and unrealistic

4. Algorithm design Largely opportunisticLargely opportunistic Computer graphics is a maturing fieldComputer graphics is a maturing field Hence, a more directed approach is neededHence, a more directed approach is needed

5. Long Term Strategy Understand the differences between natural and computer generated scenesUnderstand the differences between natural and computer generated scenes Understand the Human Visual System and how it perceives imagesUnderstand the Human Visual System and how it perceives images Apply this knowledge to motivate graphics algorithmsApply this knowledge to motivate graphics algorithms

6. This Presentation (1) Reinhard et. al., “Color Transfer between Images”, IEEE CG&A, Sept. 2001.

7. This Presentation(2) Reinhard et. al., “Photographic Tone Reproduction for Digital Images, SIGGRAPH 2002.

8. Introduction The Human Visual System is evolved to look at natural images NaturalRandom

9. Human Visual System

10. Retina

11. Color Processing Rod and Cone pigments

12. Color Processing Cone output is logarithmic Color opponent space

13. Image Statistics Ruderman’s work on color statistics:Ruderman’s work on color statistics: –Principal Components Analysis (PCA) on colors of natural image ensembles –Axes have meaning: color opponents (luminance, red-green and yellow-blue)

14. Color Processing Summary Human Visual System expects images with natural characteristics (not just color)Human Visual System expects images with natural characteristics (not just color) Color opponent space has decorrelated axes (but in practice close to independent)Color opponent space has decorrelated axes (but in practice close to independent) Color space is logarithmic (compact and symmetric data representation)Color space is logarithmic (compact and symmetric data representation) Independent processing along each axis should be possible  ApplicationIndependent processing along each axis should be possible  Application

15. L Color Space Convert RGB triplets to LMS cone space Convert RGB triplets to LMS cone space Take logarithm Take logarithm Rotate axes Rotate axes

16. Color Transfer Make one image look like anotherMake one image look like another For both images:For both images: –Transfer to new color space –Compute mean and standard deviation along each color axis Shift and scale the target image to have the same statistics as the source imageShift and scale the target image to have the same statistics as the source image

17. Why not use RGB space? Input imagesOutput images RGB L

18. Color Transfer Example

21. Color Processing Summary Changing the statistics along each axis independently allows one image to resemble a second imageChanging the statistics along each axis independently allows one image to resemble a second image If the composition of the images is very unequal, an approach using small swatches may be used successfullyIf the composition of the images is very unequal, an approach using small swatches may be used successfully

22. Tone Reproduction

24. Global vs. Local GlobalGlobal –Scale each pixel according to a fixed curve –Key issue: shape of curve LocalLocal –Scale each pixel by a curve that is modulated by a local average –Key issue: size of local neighborhood

25. Global Operators TumblinWard Ferwerda

26. Global Operators TumblinWard Ferwerda

27. Local Operator Pattanaik

28. Spatial Processing Circularly symmetric receptive fieldsCircularly symmetric receptive fields Centre-surround mechanismsCentre-surround mechanisms –Laplacian of Gaussian –Difference of Gaussians –Blommaert Scale space modelScale space model

29. Scale Space (Histogram Equalized Images)

30. Tone Reproduction Idea Modify existing global operator to be a local operator, e.g. Greg Ward’sModify existing global operator to be a local operator, e.g. Greg Ward’s Use spatial processing to determine a local adaptation level for each pixelUse spatial processing to determine a local adaptation level for each pixel

31. Blommaert Brightness Model Gaussian filter Center/surround Neural response Brightness

32. Brightness

33. Scale Selection Alternatives Mean value Thresholded How large should a local neighborhood be?

34. Mean Value

35. Thresholded

36. Tone-mapping Local adaptation Greg Ward’s tone- mapping with local adaptation

37. Results Good results, but something odd about scale selection:Good results, but something odd about scale selection: For most pixels, a large scale was selectedFor most pixels, a large scale was selected Implication: a simpler algorithm should be possibleImplication: a simpler algorithm should be possible

38. Simplify Algorithm Greg Ward’s tone- mapping with local adaptation Simplify Fix overall lightness of image

39. Global Operator Results WardOur method

40. Global Operator Results WardOur method

41. Global  Local Global operator Local operator

42. Local Operator Results Global Local

43. Local Operator Results GlobalLocalPattanaik

44. Summary Knowledge of the Human Visual System can help solve engineering problemsKnowledge of the Human Visual System can help solve engineering problems Color and spatial processing investigatedColor and spatial processing investigated Direct applications shownDirect applications shown

45. Future Work This presentation

46. Acknowledgments Thanks to my collaborators: Peter Shirley, Jim Ferwerda, Mike Stark, Mikhael Ashikhmin, Bruce Gooch, Tom TrosciankoThanks to my collaborators: Peter Shirley, Jim Ferwerda, Mike Stark, Mikhael Ashikhmin, Bruce Gooch, Tom Troscianko This work sponsored by NSF grants 97- 96136, 97-31859, 98-18344, 99-78099 and by the DOE AVTC/VIEWSThis work sponsored by NSF grants 97- 96136, 97-31859, 98-18344, 99-78099 and by the DOE AVTC/VIEWS