Theoretical and Practical Limits to Wide Color Gamut Imaging in Objects, Reproducers, and Cameras Wayne Bretl Presented at SMPTE Fall Conference October 2011

Gamut Limitations Object surface-color limits – Covered in textbooks and the paper accompanying this presentation Reproducer gamut limits – Covered thoroughly in textbooks Camera Limits – Less understood – Focus of this presentation

Can Imperfect Cameras be Perfected? Questions: – Do practical cameras cover the full visual gamut? Can they cover the full visual gamut? Should they cover the full visual gamut? Explore with Gaussian test spectra

Approach to the Question Compare the eye/perfect camera to practical cameras Explore the color space with Gaussian spectra of varying bandwidth and center wavelength that the eye can readily distinguish and compare practical camera responses to the ideal

What the Perfect Camera Does Reports the same color (X, Y, Z) as the Standard Observer for any object spectrum This happens if the camera spectral sensitivities are linear transforms of the Standard Observer – but what if they arent?

A Perfect Camera 3X3 LINEAR MATRIX Camera = sensor + transformation Camera = Perfect Camera if: sensor + transformation = Standard Observer Perfect Camera

A Perfect Camera Viewing Gaussian Test Spectra Chromaticities of Gaussian Spectra: – The spectral locus is the extreme case of infinitesimal bandwidth – For a finite bandwidth, saturation reduced in cyan due to wide L-cone response Note: Wide L-cone response also explains cyan limits in 3-color additive reproduction and lack of highly saturated cyan surface colors Gaussian Spectrum +/- 40 nm Gaussian test spectra with various widths and center wavelengths, as seen / differentiated by a perfect camera. Can practical cameras differentiate these object spectra? Gaussian Spectrum +/- 40 nm Gaussian and single wavelength cannot be distinguished single wavelength

How Are the Spectral Responses and the Spectral Locus Related ? High L response pulls the locus toward red Simultaneously decreasing L and increasing S make the spectral locus curved, even though the transform is linear Single Wavelength Input Example – Standard Observer (Wide L Response) LINEAR TRANSFORM

L and S do not overlap Hypothetical narrow L response How Are the Spectral Responses and the Spectral Locus Related ? Example – Narrow L Response LINEAR TRANSFORM The reported spectral locus is triangular because only two sensor outputs vary at once Spectral locus straight line as L and M vary Spectral locus straight line as M and S vary

Hypothetical narrow L response How Are the Spectral Responses and the Spectral Locus Related ? Example – Narrow L Response LINEAR TRANSFORM Note that evenly-spaced wavelengths produce unevenly spaced chromaticities due to the shapes of the response curves (even though the transform process is linear)

Hypothetical narrow L response Consequences for Saturated Cyan Colors Example – Narrow L Response LINEAR TRANSFORM Spectra that look different to the eye cannot be distinguished by this camera Eyeball single wavelength Eyeball Gaussian spectrum Camera single wavelength or Gaussian

Before Going Further: Does Camera Gamut Exist? (Yes!) *Jack Holm, Capture Color Analysis Gamuts, Fourteenth Color Imaging Conference: Color Science and Engineering Systems, Technologies, Applications, Scottsdale, Arizona; November 2006; p ; ISBN / ISSN: HOLMS* EXAMPLE: (solid green line) SPECTRAL LOCUS REPORTED BY A PRACTICAL SENSOR/TRANSFORM PAIR CAMERA GAMUT (LIMIT OF REPORTABLE COLORS) = CONVEX ENVELOPE ENCLOSING THE SPECTRAL LOCUS Note: the TRANSFORM used is linear, and determined by best fit to important object colors and/or most pleasing results EXAMPLE: MIX TWO WAVELENGTHS INPUT HERE REPORTED HERE MIXTURE REPORTED HERE INPUT HERE REPORTED HERE

Comparisons of Four Imperfect Cameras Spectral responses of practical devices – narrower responses than the Standard Observer (especially, narrow Red response) Different linear 3x3 matrix transform for each sensor – optimized for best fit to SMPTE 303 color test chart

Four Imperfect Cameras Digital still camera with narrow responses Transparency film with very narrow responses Prismatic optics (TV) camera Digital still camera with overlapping responses

Outline of Discussion Results for SMPTE 303 color test chart and the reported spectral locus Results for Gaussian spectra – Discussion: possibilities for using non-linear transforms (a camera profile, look-up table, or non-linear matrix)

Test Chart and Spectral Locus Results for the test chart and the reported spectral locus (i.e., locus for single wavelengths), when optimized for the test chart Also: Relationship between the camera spectral response and the reported spectral locus

Digital Still Camera with Narrow Responses A B A B Points A, B, C show how the spectral gamut is related to the spectral responses. Cyan edge is straight line in region of constant R response. Color chart results Eye Camera Eyeball spectral locus Camera reported spectral locus Pointers surface colors limit Test chart is reasonably accurate, but spectrum is not. C C RGB(D65) X= Y= Z= Spectral Responses Matrix Result

RGB(D65) X= Y= Z= Digital Still Camera with Narrow Responses A B A B Points A, B, C show how the spectral gamut is related to the spectral responses. Camera reported spectral locus C C Matrix Color chart results Eye Camera Camera reported spectral locus C B A Color chart results Eye Camera RGB(D65) X= Y= Z= The linear matrix scales and/or rotates everything in the chromaticity plot Greater coverage of the visual gamut, but all colors are distorted

Digital Still Camera with Narrow Responses Real colors that never can be reported Unreal colors that may be reported Net effect on camera gamut:

Transparency Film Sensitivities Note: Camera gamut includes gamut of IT8 test chart produced on this material. Transparency Film Sensitivities - Combined with Hypothetical Linear Matrix IT-8 Test Chart Gamut RGB(D65) X= Y= Z= Many wavelengths are reported as the same color Full analysis would include non-linearities, interlayer effects and the effects of the reproducing dyes – producing an effective matrix different from the optimum linear matrix Reported spectral locus is triangular due to non-overlapping R and B responses.

Whats Going On? Traditional imaging (film, lithography) does not offer an accessible transform matrix Narrow spectral sensitivities are used deliberately, to increase the reported saturation of ordinary test objects with relatively broad spectra. – This compensates for the limited saturation of the dyes/inks (which are equivalent to a built-in transform matrix with small coefficients) The result (not usually noted in the past): the system cannot distinguish between medium-high-saturation and very-high-saturation blue-greens. – (But the latter are uncommon in objects anyway.) This study assumes colorimetric accuracy for the test chart is desired. – Complete film systems are designed for increased contrast and saturation Compensates color appearance effects, provides preferred reproduction The limitations on sensing the differences between highly saturated object colors still exist when the output contrast and saturation are increased. Why Is the Film Gamut Apparently So Limited in Green and Cyan?

Digital Still Camera with Wider, Overlapping Responses Nose of the reported spectral curve is rounded due to overlap of R and B responses. Cyan and yellow edges are slightly curved RGB(D65) X= Y= Z=

TV Camera with Linear Matrix Reported spectral locus is triangular due to non- overlapping R and B responses. Similar to film, but requires larger matrix coefficients and therefore covers a larger gamut. (Display is assumed not to limit the gamut) RGB(D65) X= Y= Z=

Results for Gaussian Spectra The preceding slides showed results for color test chart chips (good results due to matrix design) and single-wavelength spectra (results not so good) Need to know what happens for intermediate cases – how does a practical camera differentiate among saturated colors: – Hard limiting or gradual distortion? – Can distortions be corrected? Use Gaussian spectra 1.Variable bandwidth, with fixed center wavelength 2.Variable center wavelength, with fixed bandwidth 3.Variable bandwidth and center wavelength

1.Variable Spectral Bandwidth (Saturation) 510 nm center wavelength with variable bandwidth Standard deviation (sigma): 1, 10, 25, 35, 45, 60, 80, 120 nm Look for: – Saturation distortion – Hue Distortion

Narrow-Response Camera Poor candidate for expansion by non- linear transform, because highly- saturated colors compress up against the spectral locus. 510 nm How does this linear system produce such non-linear looking results? Its due to the varying widths of the object spectra being multiplied by the curved shapes of the spectral responses Variable Bandwidth (Saturation) Series Colors outside the camera gamut Must be reported somewhere inside the camera gamut

Transparency Film Saturation/Hue Compression 510 nm 570 nm Multiple saturation levels are compressed strongly onto the reported spectral locus Variable Bandwidth (Saturation) Series: Sigma = 10, 25, 35, 45, 60, 80, 120 nm Animation: Center = 510 to 570 nm in 10 nm steps

Transparency Film Saturation/Hue Compression 510 nm 570 nm Multiple saturation levels are compressed strongly onto the reported spectral locus Variable Bandwidth (Saturation) Series: Sigma = 10, 25, 35, 45, 60, 80, 120 nm Animation: Center = 510 to 570 nm in 10 nm steps Multiple center wavelengths are compressed strongly toward the corners of the reported gamut. Poor candidate for non-linear correction

Overlapping Response Camera Compression 510 nm 570 nm Different wavelengths are not compressed as strongly towards the corners of the reported gamut as with non-overlapping responses; however, some saturation compression is present Variable Bandwidth (Saturation) Series: Sigma = 10, 25, 35, 45, 60, 80, 120 nm Animation: Center = 510 to 570 nm in 10 nm steps

Overlapping Response Camera Compression 510 nm 570 nm Different wavelengths are not compressed as strongly towards the corners of the reported gamut as with non-overlapping responses; however, some saturation compression is present Variable Bandwidth (Saturation) Series: Sigma = 10, 25, 35, 45, 60, 80, 120 nm Animation: Center = 510 to 570 nm in 10 nm steps Some non-linear correction may be possible. Different wavelengths reported as different points

Prism Optics Camera Poor candidate for expansion by non- linear transform, because more saturated colors show strong hue shift and concentration towards the corners of the triangular reported gamut 510 nm Variable Bandwidth (Saturation) Series

2.Variable Center Wavelength Sigma = 40 nm with variable center wavelength 40 nm series encloses the Pointer colors Observe: – Compression onto the reported spectral locus – Distance between 40-nm locus and spectral locus indicates variation available for non-linear gamut expansion

Narrow-Response Camera Difference between single wavelength and Gaussian spectrum (eyeball) 40 nm colors (eyeball) Reported Difference between single wavelength and Gaussian spectrum (camera) Reported 40 nm colors (camera) Gaussian spectra with different center wavelengths are separated from each other Good hue discrimination, poor saturation discrimination – not a good candidate for non-linear saturation correction; hue correction is possible

Transparency Film Strong compression onto spectral locus, plus compression of colors towards corners. Poor candidate for expansion by non-linear transform Difference (eyeball) Reported difference (camera)

Overlapping Response Camera Some expansion by non-linear transform may be possible due to distance between reported 40 nm locus and reported spectral locus. Difference (eyeball) Reported difference (camera)

Prism Optics Camera Strong hue distortion and/or compression onto spectral locus. Poor candidate for expansion by non-linear transform. Note, however, that most of Pointers colors are covered without expansion. Difference (eyeball) Reported difference (camera)

3.Variable Wavelength and Bandwidth Over-all indication of compression of colors to the inside of the camera gamut Chromaticities of Gaussian Spectra with Standard Observer

Narrow-Response Camera Strong compression onto spectral locus - poor candidate for expansion by non-linear transform. Standard Observer

Transparency Film Strong compression onto spectral locus - poor candidate for expansion by non-linear transform. Standard Observer

Overlapping Response Camera Some compression onto spectral locus - candidate for expansion by non-linear transform down to sigma = approx. 30 nm? Note Pointers colors are covered without expansion. Standard Observer

Prism Optics Camera Strong compression onto spectral locus - poor candidate for expansion by non-linear transform. Note: most of Pointers colors are covered without expansion. Standard Observer

TV Camera with NTSC Display, Without Matrixing Effective matrix is determined by display primaries (no electrical matrix) Performance for high saturation Gaussian spectra is similar to the same sensor with a linear matrix and no display limitations Color chart accuracy is somewhat reduced – Errors are mainly in saturation – Somewhat alleviated by customer adjustment of the receiver color controls RGBD65 X= Y= Z=

TV Camera with NTSC Display, Without Matrixing

Matrix with large coefficients is required (chroma noise increased) Junction-Depth Sensor Spectral response curves are broadly overlapping LMS(D65) X= Y= Z= Capable of distinguishing highly saturated colors; good candidate for non-linear transform

Junction-Depth Sensor

Thoughts About Camera Gamut - Three-Color Subtractive Reproduction Three-Color Additive Reproduction Object colors and reproducer gamuts are somewhat limited Surface Colors The camera may not have to be perfect over the full visual gamut But also consider that archived material might be used with future wide-gamut systems Consider: How Big a Gamut is Needed?

Conclusions A camera (sensor + transform) typically has a color gamut that limits the cyan scene colors that it can report Most cameras have a smaller gamut than the human visual system Sensors with narrow responses increase the reported saturation of test chart colors but also limit the gamut of saturated colors that can be reported Many systems developed in the past had a camera gamut that limited green and cyan saturation, and had a strong distortion of spectral colors, but were subjectively of very high quality, and were highly successful Cameras with sufficiently wide responses can have a gamut including Pointers surface colors, and excluding only colors seldom or never encountered in natural scenes: – The spectrum – Lasers – LEDs – Gas discharge tubes? – Back-lit stained glass? – Blue-green Jell-O?

Conclusions Gamut expansion (actually distinguishing among saturated colors) is impractical when camera spectral responses are too narrow – Requires impractically strong non-linear correction for chromaticities outside the reported spectral gamut – Danger of distorting colors of important common objects Some gamut expansion may be possible when there is sufficient overlap of the camera spectral responses, but: – Overlapping spectral responses require larger conversion coefficients, increasing colored noise – Expansion much beyond the reported spectral gamut (by non-linear means) may not be needed if the reported gamut is sufficiently large Accurately covering (Pointers) surface colors and a bit more is a possible practical compromise – Combination of some sensor spectral response overlap and some non-linear processing

References Holm, Jack, Capture Color Analysis Gamuts, Fourteenth Color Imaging Conference: Color Science and Engineering Systems, Technologies, Applications, Scottsdale, Arizona; November 2006; p ; ISBN / ISSN: Buil, Christian, CANON 40D, 50D, 5D, 5D Mark II Comparison, Eastman Kodak Company Publication no. E-88, Technical Data / Color Reversal Film, Kodachrome 64 and 200 Films, June Ballard, Jay, TK-41 prism spectral measurements (private communication) SMPTE, Standard 0303M-2002, Television – Color Reference Pattern IT8.7/ (Reaffirmed 2008) Graphic technology - Color transmission target for input scanner calibration Pointer, M.R., The gamut of real surface colors, Color Research and Application, 5, pp. 145–155, International Telecommunication Union Radiocommunication Sector, Recommendation ITU-R BT.709-5, Parameter values for the HDTV standards for production and international programme exchange, BT I!!PDF-E.pdfhttp:// BT I!!PDF-E.pdf Hunt, R.W.G., The Reproduction of Color, Sixth Edition, John Wiley & Sons Ltd, 2004, reprinted March 2006, Chapters 7 and 9 Ibid., sections 9.4 and 9.5, pp Ibid., section 35.4, pp Ibid., section 2.5, pp Ibid., section 9.2, pp

Thank You