1 Invariant Image Improvement by sRGB Colour Space Sharpening 1 Graham D. Finlayson, 2 Mark S. Drew, and 2 Cheng Lu 1 School of Information Systems, University of East Anglia Norwich (U.K.) 2 School of Computing Science, Simon Fraser University Vancouver (CANADA)

2 What is an invariant image? We would like to obtain a greyscale image which removes illuminant effects.

3 Shadows stem from what illumination effects? Changes of illuminant in both intensity and colour Intensity – can be removed in chromaticity space Colour – ? shadows still exist in the chromaticity image! Region Lit by Sky-light only Region Lit by Sunlight and Sky-light

4 Model of illuminants Illumination is restricted to the Planckian locus represent illuminants by their equivalent Planckian black-body illuminants Wien’s approximation: Most typical illuminants lie on, or close to, the Planckian locus

5 Image Formation Camera responses depend on 3 factors: light (E), surface (S), and sensor (Q)  is Lambertian shading

6 Q 2 ( ) Sensitivity Q 1 ( )Q 3 ( ) = Delta functions “select” single wavelengths: Using Delta-Function Sensitivities

7 For delta-function sensors and Planckian illumination we have: Back to the image formation equation Surface Light

8 Band-ratio chromaticity R G B Plane G=1 Perspective projection onto G=1 Let us define a set of 2D band-ratio chromaticities: p is one of the channels, (Green, say) [or Geometric Mean]

9 Let’s take log’s: Band-ratios remove shading and intensity with Gives a straight line: Shading and intensity are gone.

10 Calibration: find invariant direction Log-ratio chromaticities for 6 surfaces under 14 different Planckian illuminants, HP912 camera Macbeth ColorChecker: 24 patches

11 Deriving the Illumination Invariant This axis is invariant to shading + illuminant intensity/colour

12 Algorithm, cont’d: Form greyscale I’ in log-space: exponentiate: Finlayson et al.,ECCV2002

13 Problems in Practice What if camera sensors are not narrowband? Find a sensor transform M that sharpens camera sensors Equivalent to transforming RGB to a new colour space Kodak DCS420 camera sensors  3 x 3 colors

14 Problem 2: Nonlinearity We generally have nonlinear image data.  Linearise images prior to invariant image formation Forming invariant image from nonlinear images

15 Approach : solve for sharpened sRGB space sRGB – standard RGB Color Management strategy proposed by Microsoft and HP A device independent color space – small cost for storage and transfer Transform CIE tristimulus values so as to suit to current monitors XYZsRGB

16 sRGB-to-XYZ conversion Two steps: Nonlinear sRGB to linear RGB –Gamma correction Transformation to CIE XYZ tristimulus with a D65 white point –Using a 3 by 3 matrix M The problem of nonlinearity solved ! (well enough) The problem of non-narrowband sensors XYZ D65 color matching functions are quite sharp, but can be sharper.

17 Spectral sharpening for XYZ D65 ∴ Apply database spectral sharpening mapping two sets of patch images formed with the camera under two different lights, with a 3 x 3 matrix P For diagonal color constancy, compute eigenvectors T of P The sharpened XYZ color matching functions under D65 have narrower curves.

18 Linear sRGB color space sharpening Concatenating the conversion to the XYZ tristimulus values by the spectral sharpening transform T: a sharpened sRGB space. Performing the invariant image finding routine in this new sharpened linear color space: RGB → sRGB → XYZ → XYZ # SMT

19 One more trick Logarithms of colour ratios in finding the invariant involves a singularity Modify by making use of a generalised logarithm function:

20 Some examples