1. Color Dr. Craig Reinhart

2. What is Color? Color is merely a concept, something we “see” within our minds –It’s interpretation involves both physics and biology –How would you describe the color “red” to a blind person? Clearly, it plays a useful role in everyday life Thus, building a mathematical description of color may also prove useful

3. Color is Complex “Standard” mathematical models began in the early 20 th century and have evolved (and evolved, and evolved…) Confusion arises in that the early standards are not discarded as the evolution takes place –Today, “old” and “new” standards live side by side Thus, when discussing color the first thing the participants must agree upon is the standard in which they are basing their discussion

4. The Standards Based on a tristimulus system of additive primaries Tristimulus – three primary colors Additive – all other colors can be created by adding different proportions of the primaries

5. Preliminaries

6. Tristimulus, Additive Primaries Red, Green, and Blue primaries were agreed upon based on a normal human visual system A normal visual system consists of the eyes and sections of the brain, all operating properly –Color blindness is due to a deficiency in one type of cone – very common in males Red and green receptor genes are carried on the X chromosome, and these are the ones that typically go wrong Women need two bad X chromosomes to have a deficiency, which is less likely

7. The Eye

8. The Retina The retina contains two types of light sensors –Rods that are highly sensitive to light and provide us with “night vision” Located primarily in the outer (non-foveal) region of the retina –Cones that are highly sensitive to color and provide us with “color vision” Located primarily in the central (foveal) region of the retina Are adaptive to ambient light Are susceptible to optical illusions Color illusion

9. The Retina There are 3 types of cones contained within the retina –Red-sensitive (long) –Green-sensitive (medium) –Blue-sensitive (short)

10. Cone Sensitivity (probable)

11. The Visual System Once the eye has sensed the color it is up to the brain to interpret it This is where things get very complex and relatively little is known about the actual inner-workings

12. The Visual System

13. So What? With what we know (or think we know) about the visual system, we now try to develop useful models to support the more mundane tasks of everyday life

14. The Standards

15. Standard Observers To set a standard a group of people were shown color patches of a given size and asked “what colors they saw” –Match color by adjusting primaries Results were averaged and thus the standards were created

16. Standard Observers 1931 (2°) and 1964 (10°) standard observers

17. The Color Spaces Mathematical Descriptions of Color

18. CIE Color Spaces Used primarily for matching/comparing colors Various different forms of charts Charts were made using “standard observers” –Groups of people with “normal” color vision –Ties wavelengths to colors –Can specify coordinates to compensate for monitor characteristics There are numerous versions of the CIE color space based on differing observer parameters and differing basis standards

19. XYZ Color Space (the grand-daddy of them all) Combine –Known illuminant –Colors on known (non-reflective) material –Standard observer –The result is a tristimulus space for describing colors

20. xyY Color Space (the first offspring) There’s no good way to visualize the XYZ color space The xyY space is a normalized version of XYZ –x and y correspond to normalized X and Y respectively –The luminance (black/white level) is lost in the normalization process so Y (in xyY) is also computed from XYZ –z is not needed since the normalization process constrains x + y + z = 1

21. xyY Color Space (well, one of them anyway) Monochromatic (saturated) Colors Monitor Gamut Planckian (blackbody) Locus Line of Purples (not monochromatic)

22. xyY Color Space Pro –We can visualize the proximity of one color to another Con –The space is non-uniform so we cannot use it to compare colors

23. Other Useful Color Spaces What do we know? –Color spaces should be tristimulus –XYZ and xyY are not very intuitive –We need something to suite our [varied] needs So, we invent new color spaces to suit our needs

24. RGB Color Space RGB is a linear color space –Pure red, green, and blue are the basis vectors for the space –Useful for cameras, monitors, and related manipulations Gray (black to white) axis Black White

25. RGB Color Space Back Surfaces Front Surfaces

26. RGB Operations Color mixing is performed by vector addition and subtraction operations –Adding/subtracting colors is the same as adding/subtracting vectors (with clamping at 0) redgreen + yellow =

27. RGB Operations Increasing or decreasing luminance is performed by scalar multiplication –Same as scalar multiplication of vectors (with clamping at some maximum) yellow2 * brighter yellow =

28. RGB Operations A word of caution… Operations must be clamped… –…at 0 to make sure components don’t go negative –…at some pre-specified maximum to ensure display compatibility Scaling down from a value greater than the allowed maximum can be performed but care must be taken –Bright colors may end up less bright than other colors in the scene –The answer is to scale ALL colors in the scene which can be expensive

29. RGB Color Space RGB RedGreenBlue

30. RGB Color Space Pro –Very intuitive and easy to manipulate when generating colors Con –Very unintuitive when it comes to comparing colors Consider the Euclidian distance between red and green and between green and blue

31. Luminance-Chrominance Color Spaces (there are many) Luminance channel –Corresponds to the black and white signal of a color television Two chrominance channels –Red and blue –Correspond to the color signal that “rides” on top of the black and white signal of a color television Various forms –YUV, YIQ, YC b C r, YP b P r …

32. Luminance-Chrominance Color Spaces (there are many) Luminance is a square wave Chrominance is a sine wave (modulation) on top of the square wave

33. Luminance-Chrominance Color Spaces (there are many) Simple conversion from RGB and YPbPr And from YPbPr to RGB

34. Luminance-Chrominance Color Spaces RGB Luminance Chrominance Blue Chrominance Red

35. Luminance-Chrominance Color Spaces Pro –Separate high frequency components from low frequency components –Easy to compute –Facilitates image compression (JPEG, MPEG) Con –Not very intuitive –Require signed, floating point (or scaled) representation –Multiple forms causes confusion (e.g. people regularly confuse YCbCr with YUV)

36. Luminance-Chrominance Color Spaces Note that there are various different matrices for these conversions –Based on different needs –Be careful about the one you select Chrominance channels are +/- so to display you must translate and scale

37. Compression (uses for luminance/chrominance) Trade-off between the amount of data and the quality of the picture –Throw away as much data as possible without degrading the picture –JPEG, MPEG, …

38. JPEG/MPEG The edge/structure detail is contained in the luminance channel –This is referred to as “high frequency” data The color information is in the chrominance channels which are lacking edges/structure detail –This is referred to as “low frequency” data

39. Color Image (RGB)

40. Y Channel (high frequency)

41. Cb Channel (low frequency)

42. Cr Channel (low frequency)

43. Subsampling By subsampling we achieve a 2:1 compression without doing any “work” –This is the default mode for MPEG –The default mode for JPEG is to subsample in 1 dimension only so it’s 3:2 compression without doing any “work” The decompressed image still looks good because of the low frequency nature of the chrominance channels

44. Subsample Cb and Cr (mpeg mode)

45. MPEG/JPEG There’s a lot more processing involved but they’re not specific to the chosen color space

46. Cyan-Magenta-Yellow-blacK Used in printing Colored pigments (inks) remove color from incident light that is reflected off of the paper CMYK is a subtractive set of primaries –K (Black) is not actually necessary but is added for practical printing applications CMYK is a linear color space

47. Cyan-Magenta-Yellow-blacK Cyan MagentaYellowBlack RGB

48. Cyan-Magenta-Yellow-blacK Pro –Good for printing (as long as you include the K ink) Con –Difficult to convert from RGB to CMYK as it is not a simple subtraction from white like much of the world would lead you to believe

49. Hue/Saturation/Lightness Also Hue/Saturation/Value or Hue/Saturation/Intensity Suitable to processing images for “human consumption” (viewing) –Easy to make colors more “vibrant” (and other features that we can name but can’t really describe) –Used in artistic endeavors

50. Hue/Saturation/Lightness Hue is the pure color content –Corresponds to the edges of the RGB cube Saturation is the intensity of color –The faces of the RGB cube are fully-saturated Lightness is the brightness of the color –Ranges from black to white

51. Hue/Saturation/Lightness Mapping the RGB cube to a hex-cone

52. Hue/Saturation/Lightness RGB Hue Saturation Lightness

53. Hue/Saturation/Lightness Pro –Captures the “human” qualities of color Con –Difficult to describe –Difficult to compute

54. L*a*b* Color Space While convenient for various reasons, the previous color spaces are not great for comparing colors –Most attempts treat the colors as a 3-vector and try to do some modified Euclidian distance measure and some sort of clustering algorithm –But, the color spaces are non-uniform La*b* is a uniform color space –A small perturbation in a color component is equally perceptible across the entire range

55. L*a*b* Color Space RGB L* a* b*

56. L*a*b* Color Space Pro –Uniform space –Colors can be compared [accurately] using the Euclidian distance formula Con –Not very intuitive –Not easy to convert from/to RGB Requires knowledge of a reference white Requires computation of cube-roots In general, not all that useful for computer graphics applications

57. Summary Color is complex The human visual system is complex and very good at processing light Together they comprise a system that we aren’t even close to understanding but utilize very effectively

58. Other Related Topics And what good talk on color would dare to leave out these topics…

59. The GretagMacBeth TM ColorChecker®

60. The JOBO Card

61. Gamma RGB values from a camera (for instance) are linear RGB values viewed on a monitor are non-linear Gamma correction is a non-linear pre-adjustment of the linear RGB values to match (or meet the expectations of) the non-linear human visual system when viewing a non-linear monitor Implemented as a look-up table 0.01.0 0.0 R’G’B’ RGB

62. Gamma Correction Linear RGB from camera Uncorrected Linear RGB on monitor Corrected Linear RGB from camera Corrected Linear RGB on monitor

63. Alpha In computer graphics, we often speak of 32- bit RGB The additional 8-bits is not another color basis, but rather a value called Alpha Alpha defines how colors combine with one another in an operation called Alpha Blending

64. Alpha Blending In 3D computer graphics objects naturally obscure other objects Depending on the make-up of the object in front –You may not see the object in back, the object in front is opaque –You may only see the object in back, the object in front is translucent –You may see some combination of both objects

65. Alpha Blending The specification of an objects opacity is done through alpha The basic formula is one of linear interpolation The alpha value of the object in back is ignored In the event that we have multiple objects stacked, then the z-buffer rendering performs this calculation in order, back to front

66. Color Space Quantization There are times (used to be times?) when our hardware does not (did not?) support 2 24 (24-bit) colors The alternative is (was?) typically an 256 (8-bit) color palette system The question then arises as to which 256 colors we should choose

67. Color Space Quantization The popularity algorithm prescribes that we select the 256 most frequently used colors in the scene we are displaying Create a histogram of all 2 24 possible colors Keep only the top 256

68. Popularity Algorithm COUNT COLOR INDEX Color Frequency Histogram Create color frequency histogram Sort histogram by count Keep the 256 colors with the largest counts Convert all other scene colors to the closest kept color

69. Popularity Algorithm This algorithm works fine for a small amount of original scene colors (relative to the target number of colors) When the number of different colors in the original scene is much greater than the target number, the algorithm breaks down –Especially where small scene objects are concerned

70. Median-Cut Algorithm Rather than just histogram and keep the most popular colors, the median-cut algorithm attempts to find colors that represent equal numbers of colors in the original scene –Map color pixels onto the color cube –Recursively split the cube into smaller cubes, attempting to keep the number of pixels in each cube the same –The procedure ends when n (the target number of colors) cubes are created –The centroids of the cubes are the retained colors –All other pixel colors in each cube are set to the cube centroid color

71. Quantization But what happens when the number of available colors is only 2? (monochrome display device) Popularity and median-cut algorithms won’t produce suitable results in this scenario

72. Threshold If we merely select a threshold and set pixel values below it to black and above it to white we lose a lot of information

73. Threshold The human visual system is so good that we can still see the picture (in our minds) even though the data (taken in by the eyes) is minimal

74. Dither By replacing individual pixels with a pattern of binary values, the human visual system can be fooled into seeing shades The problem with pure thresholding is that all of the error ends up in the pixel being processed With dithering, we attempt do distribute the error to surrounding pixels

75. Floyd-Steinberg Dithering For each pixel display the closest available color compute error = actualColor – displayedColor spread error over (weighted addition) neighboring actual pixels to the right and below 7 * error 16 1 * error 16 5 * error 16 3 * error 16 Current pixel

76. Floyd-Steinberg Dithering Again, the human visual system is so good that we can still see the picture (in our minds) even though the data (taken in by the eyes) is minimal

77. Dithering Other dithering techniques involve replacing pixels with patterns meant to approximate the amount of “ink” (intensity) on the page The downside of these approaches are that the display size is typically larger than the actual image

78. [a few selected] References How the Retina Works – Helga Kolb American Scientist, Volume 91 Calculation From the Original Experimental Data of the CIE 1931 RGB Standard Observer Spectral Chromaticity Coordinates and Color Matching Functions – D.A. Broadbent University de Sherbrooke Eye, Brain, and Vision – David H. Hubel Scientific American Library 1988 RGB Coordinates of the Macbeth ColorChecker – Danny Pascale www.babelcolor.com The RGB Code: The Mysteries of Color Revealed – Danny Pascale www.babelcolor.com

79. Programming Assignment (part 1) When you tell WindowsXP to shut down, if you do not confirm right away the desktop scene fades from color to gray scale Write two programs –Use the RGB color space to fade from color to gray –Use the YPbPr color space to fade from color to gray Create a video for each showing the fade from color to gray Demonstrate each approach in class (show videos) Describe each of the two approaches used Due next class period

80. Programming Assignment (part 2) Implement Floyd-Steinberg dithering –Run on a gray scale image (R=G=B) Implement “pattern” dithering –Run on a gray scale image (R=G=B) Use one of your fade processes from part 1 to fade your input gray scale image to your Floyd-Steinberg dithered image (see next page for example)

81. RGB→YP b P r YP b P r →YYY Fade algorithm creating intermediate files for video Conversions to set source image Fade algorithm to from source to target YYY→Dithered (Binary) Floyd-Steinberg to set target