1. Why is this hard to read

3. Unrelated vs. Related Color Unrelated color: color perceived to belong to an area in isolation (CIE 17.4) Related color: color perceived to belong to an area seen in relation to other colors (CIE 17.4)

4. Illusory contour Shape, as well as color, depends on surround Most neural processing is about differences

5. Illusory contour

6. CS 768 Color Science Perceiving color Describing color Modeling color Measuring color Reproducing color

8. Spectral measurement Measurement p( ) of the power (or energy, which is power x time ) of a light source as a function of wavelength Usually relative to p(560nm) Visible light 380-780 nm

11. Retinal line spread function retinal position relative intensity

12. Linearity additivity of response (superposition) r(m 1 +m 2 )=r(m 1 )+r(m 2 ) scaling (homogeneity) r(  m)=  r(m) r(m 1 (x,y)+m 2 (x,y))= r(m 1 )(x,y)+r(m 2 )(x,y)= (r(m 1 )+r(m 2 ))(x,y) r(  m(x,y))=  r(m)(x,y) retinal intensity monitor intensity

13. Non-linearity

15. http://webvision.med.utah.edu/

16. Ganglion Bipolar Amacrine Rod Cone Epithelium Optic nerve Retinal cross section Light Horizontal

17. Visual pathways Three major stages –Retina –LGN –Visual cortex –Visual cortex is further subdivided http://webvision.med.utah.edu/Color.html

18. Optic nerve 130 million photoreceptors feed 1 million ganglion cells whose output is the optic nerve. Optic nerve feeds the Lateral Geniculate Nucleus approximately 1-1 LGN feeds area V1 of visual cortex in complex ways.

19. Photoreceptors Cones - –respond in high (photopic) light –differing wavelength responses (3 types) –single cones feed retinal ganglion cells so give high spatial resolution but low sensitivity –highest sampling rate at fovea

20. Photoreceptors Rods –respond in low (scotopic) light –none in fovea try to foveate a dim star—it will disappear –one type of spectral response –several hundred feed each ganglion cell so give high sensitivity but low spatial resolution

21. Luminance Light intensity per unit area at the eye Measured in candelas/m 2 (in cd/m 2 ) Typical ambient luminance levels (in cd/m 2 ): –starlight 10 -3 –moonlight 10 -1 –indoor lighting 10 2 –sunlight 10 5 –max intensity of common CRT monitors 10 ^2 From Wandell, Useful Numbers in Vision Science http://white.stanford.edu/~brian/numbers/numbers.html

22. Rods and cones Rods saturate at 100 cd/m 2 so only cones work at high (photopic) light levels All rods have the same spectral sensitivity Low light condition is called scotopic Three cone types differ in spectral sensitivity and somewhat in spatial distribution.

23. Cones L (long wave), M (medium), S (short) –describes sensitivity curves. “Red”, “Green”, “Blue” is a misnomer. See spectral sensitivity.

25. Receptive fields Each neuron in the visual pathway sees a specific part of visual space, called its receptive field Retinal and LGN rf’s are circular, with opponency; Cortical are oriented and sometimes shape specific. - - - - - - -- - + - - On center rfRed-Green LGN rf + + + + + + + + - - - Oriented Cortical rf

27. Channels: Visual Pathways subdivided Channels Magno –Color-blind –Fast time response –High contrast sensitivity –Low spatial resolution Parvo –Color selective –Slow time response –Low contrast sensitivity –High spatial resolution Video coding implications Magno –Separate color from b&w –Need fast contrast changes (60Hz) –Keep fine shading in big areas –(Definition) Parvo –Separate color from b&w –Slow color changes OK (40 hz) –Omit fine shading in small areas –(Definition) (Not obvious yet) pattern detail can be all in b&w channel

29. Trichromacy Helmholtz thought three separate images went forward, R, G, B. Wrong because retinal processing combines them in opponent channels. Hering proposed opponent models, close to right.

30. Opponent Models Three channels leave the retina: –Red-Green (L-M+S = L-(M-S)) –Yellow-Blue(L+M-S) –Achromatic (L+M+S) Note that chromatic channels can have negative response (inhibition). This is difficult to model with light.

31. +- +

34. 100 10 1.0 0.1 0.001 012 Log Spatial Frequency (cpd) Contrast Sensitivity Luminance Red-Green Blue-Yellow

35. Color matching Grassman laws of linearity: (     )(   (   (   Hence for any stimulus s( ) and response r( ), total response is integral of s( ) r( ), taken over all or approximately  s( )r( )

36. Primary lights Test light Bipartite white screen Surround field Test lightPrimary lights Subject Surround light

37. Color Matching Spectra of primary lights s 1 ( ), s 2 ( ), s 3 ( ) Subject’s task: find c 1, c 2, c 3, such that c 1 s 1 ( )+c 2 s 2 ( )+c 3 s 3 ( ) matches test light. Problems (depending on s i ( )) –[c 1,c 2,c 3 ] is not unique (“metamer”) –may require some c i <0 (“negative power”)

38. Color Matching Suppose three monochromatic primaries r,g,b at 645.16, 526.32, 444.44 nm and a 10° field (Styles and Burch 1959). For any monochromatic light t( ) at  find scalars R=R(  G=G(  B=B(  such that t( ) = R(  r  G(  g  B(  b R( ,  G( ,  B(  are the color matching functions based on r,g,b.

40. Color matching Grassman laws of linearity: (     )(   (   (   Hence for any stimulus s( ) and response r( ), total response is integral of s( ) r( ), taken over all or approximately  s( )r( )

41. Color matching What about three monochromatic lights? M( ) = R* R ( ) + G* G ( ) + B* B ( ) Metamers possible good: RGB functions are like cone response bad: Can’t match all visible lights with any triple of monochromatic lights. Need to add some of primaries to the matched light

42. Primary lights Test light Bipartite white screen Surround field Test lightPrimary lights Subject Surround light

43. Color matching Solution: CIE XYZ basis functions

45. Color matching Note Y is V( ) None of these are lights Euclidean distance in RGB and in XYZ is not perceptually useful. Nothing about color appearance

46. XYZ problems No correlation to perceptual chromatic differences X-Z not related to color names or daylight spectral colors One solution: chromaticity

47. Chromaticity Diagrams x=X/(X+Y+Z) y=Y/(X+Y+Z) z=Z/(X+Y+Z) Perspective projection on X-Y plane z=1-(x-y), so really 2-d Can recover X,Y,Z given x,y and on XYZ, usually Y since it is luminance

48. Chromaticity Diagrams No color appearance info since no luminance info. No accounting for chromatic adaptation. Widely misused, including for color gamuts.

49. Some gamuts SWOP ENCAD GA ink

52. MacAdam Ellipses JND of chromaticity Bipartite equiluminant color matching to a given stimulus. Depends on chromaticity both in magnitude and direction.

54. MacAdam Ellipses For each observer, high correlation to variance of repeated color matches in direction, shape and size –2-d normal distributions are ellipses –neural noise? See Wysecki and Styles, Fig 1(5.4.1) p. 307

55. MacAdam Ellipses JND of chromaticity –Weak inter-observer correlation in size, shape, orientation. No explanation in Wysecki and Stiles 1982 More modern models that can normalize to observer?

56. MacAdam Ellipses JND of chromaticity –Extension to varying luminence: ellipsoids in XYZ space which project appropriately for fixed luminence

57. MacAdam Ellipses JND of chromaticity –Technology applications: Bit stealing: points inside chromatic JND ellipsoid are not distinguishable chromatically but may be above luminance JND. Using those points in RGB space can thus increase the luminance resolution. In turn, this has appearance of increased spatial resolution (“anti-aliasing”) Microsoft ClearType. See http://www.grc.com/freeandclear.htm and http://www.ductus.com/cleartype/cleartype.html

58. CIELab L* = 116 f(Y/Y n )-16 a* = 500[f(X/X n ) – f(Y/Y n )] b* = 200[f(Y/Y n ) –f(Z/Z n )] where X n,Y n,Z n are the CIE XYZ coordinates of the reference white point. f(z) = z 1/3 if z>0.008856 f(z)=7.787z+16/116 otherwise L* is relative achromatic value, i.e. lightness a* is relative greenness-redness b* is relative blueness-yellowness

59. CIELab L* = 116 f(Y/Y n )-16 a* = 500[f(X/X n ) – f(Y/Y n )] b* = 200[f(Y/Y n ) –f(Z/Z n )] where X n,Y n,Z n are the CIE XYZ coordinates of the reference white point. f(z) = z 1/3 if z>0.008856 f(z)=7.787z+16/116 otherwise

60. CIELab L* = 116 f(Y/Y n )-16 a* = 500[f(X/X n ) – f(Y/Y n )] b* = 200[f(Y/Y n ) –f(Z/Z n )] where X n,Y n,Z n are the CIE XYZ coordinates of the reference white point. f(z) = z 1/3 if z>0.008856 f(z)=7.787z+16/116 otherwise C* ab = sqrt(a* 2 +b* 2 ) corresponds to perception of chroma (colorfulness). hue angle h ab =tan -1 (b*/a*) corresponds to hue perception. L* corresponds to lightness perception Euclidean distance in Lab space is fairly correlated to color matching and color distance judgements under many conditions. Good correspondence to Munsell distances.

61. a*>0 redder a*<0 greener b*>0 yellower b*<0 bluer chroma hue lightness

62. Complementary Colors c1 and c2 are complementary hues if they sum to the whitepoint. Not all spectral (i.e. monochromatic) colors have complements. See chromaticity diagram. See Photoshop Lab interface.

63. CIELab defects Perceptual lines of constant hue are curved in a*-b* plane, especially for red and blue hues (Fairchiled Fig 10.5) Doesn’t predict chromatic adaptation well without modification Axes are not exactly perceptual unique r,y,g,b hues. Under D65, these are approx 24°, 90°,162°,246° rather than 0°, 90°, 180°, 270° (Fairchild)

64. CIELab color difference model  E*=sqrt(  L* 2 +  a* 2 +  b* 2 ) –May be in the same L*a*b* space or to different white points (but both wp’s normalized to same max Y, usually Y=100). –Typical observer reports match for  E* in range 2.5 – 20, but for simple patches, 2.5 is perceptible difference (Fairchild)

65. Viewing Conditions Illuminant matters. Fairchild Table 7-1 shows  E* using two different illuminants. Consider a source under an illuminant with SPD T( ). If color at a pixel p has spectral distribution p(  and reflectance factor of screen is r(  then SPD at retina is r( )T( )+p( ). Typically r(  is constant, near 1, and diffuse.

66. Color ordering systems Want system in which finite set of colors vary along several (usually three) axes in a perceptually uniform way. Several candidates, with varying success –Munsell Spectra available at Finnish site –NCS –OSA Uniform Color Scales System –…–…

67. Color ordering systems CIE L*a*b* still not faithful model, e.g. contours of constant Munsell chroma are not perfect circles in L*a*b* space. See Fairchild Fig 10-4, Berns p. 69.

68. Effect of viewing conditions Impact of measurement geometry on Lab –Need illumination and viewing angle standards –Need reflection descriptions for opaque material, transmission descriptions for translucent

69. Reflection geometry diffuse specular

70. Reflection geometry Semi-glossy glossy

71. Reflection geometry Semi-glossy glossy

72. Some standard measurement geometries d/8:i diffuse illumination, 8° view, specular component included d/8:e as above, specular component excluded d/d:i diffuse illumination and viewing, specular component included 45/0 45° illumination, 0° view

73. Viewing comparison L*C*h EE d/8:i51.141.5269 45/044.846.92688.3 d/8:e47.544.62684.7 Measurement differences of a semi-gloss tile under different viewing conditions (Berns, p. 86).  E is vs. d/8:i. Data are for Lab.

74. L*u*v* CIE u' v' chromaticity coordinates: u'=4X/(X+15Y+3Z)= 4x/(-2+12y+3) v'=9Y/(X+15Y+3Z)=9y/(-2+12y+3) Gives straighter lines of constant Munsell chroma (See figures on p. 64 of Berns). L* = 116(Y/Y n ) 1/3 – 16 u* = 13L*(u' – u n ') v* = 13L*(v'-v n ')

75. L*u*v* L* = 116(Y/Y n ) 1/3 – 16 u* = 13L*(u' – u n ') v* = 13L*(v'-v n ') u n ', v n ' values for whitepoint

76. Models for color differences Euclidean metric in CIELab (or CIELuv) space not very predictive. Need some weighting  V = (1/k E) )[(  L*)/k L S L ) 2 +(  C  */k C S C ) 2 +(  H  */k H S H ) 2 ] 1/2  = uv or ab according to whether using L*a*b* or L*u*v* The k's are parameters fit to the data. The S's are functions of the underlying variable, estimated from data.

77. Models for color differences  E* 94 k L = k C = k H = 1 S L = 1 S C =1+.0.045C* ab S H = 1+0.015C* ab Fitting with one more parameter for scaling gives good predictions. Berns p 125.

78. Color constancy Color difference models such as previous have been used to predict color inconstancy under change of illumination. Berns p. 214.

79. Other color appearance phenomena Models still under investigation to account for: –Colorfulness (perceptual attribute of chroma) increases with luminance ("Hunt effect") –Brightness contrast (perceptual attribute of lightness difference) increases with luminance –Chromatic adaptation

80. Color Gamuts Gamut: the range of colors that are viewable under stated conditions Usually given on chromaticity diagram –This is bad because it normalizes for lightness, but the gamut may depend on lightness. –Should really be given in a 3d color space –L*a*b* is usual, but has some defects to be discussed later

81. Color Gamut Limitations 1.CIE XYZ underlies everything –this permits unrealizable colors, but usually "gamut" means restricted to the visible spectrum locus in chromaticity diagram 2.Gamut can depend on luminance –usually on illuminant relative luminance, i.e. Y/Y n

82. Color Gamut Limitations Surface colors –reflectance varies with gloss. Generally high gloss increases lightness and generally lightness reduces gamut (see figures in Berns, p. 145 ff) Stricter performance requirements often reduce gamut –e.g. require long term fade resistance

83. Color Gamut Limitations Physical limitations of colorants and illuminants –Specific set of colorants and illuminants are available. For surface coloring we can not realize arbitrary XYZ values even within the chromaticity spectral locus Economic factors –Color may be available but expense not justified

84. Color mixing Suppose a system of colorants (lights, inks,…). Given two colors with spectra c 1 ( ) and c 2 ( ). This may be reflectance spectra, transmittance spectra, emission spectra,…Let d be a mix of c 1 and c 2. The system is additive if d( ) = c 1 ( ) + c 2 ( ) no matter what c 1 and c 2 are.

85. Scalability Suppose the system has some way of scaling the intensity of the color by a scalar k. Examples: –CRT: increase intensity by k. –halftone printing: make dots k times bigger –colored translucent materials: make k times as thick If c is a color, denote the scaled color as d. If the spectrum d (  is k(c( )) for each  the system is scalable

86. Scalability Consider a color production system and a colors c 1,c 2 with c 2 =kc 1. Let m i =max(c i ( )) and d i =(1/m i )c i. Highschool algebra shows that the system is scalable if and only if d 1 ( )=d 2 ( ) for all, no matter what c 1 and k.

87. Control in color mixing systems Normally we control some variable to control intensity: –CRT voltage on electron gun integer 0...255 –Translucent materials (liquids, plastics...): thickness –Halftone printing: dot size

88. Linearity A color production system is linear if it is additive and scalable. Linearity is good: it means that model computations involving only linear algebra make good predictions. Interesting systems are typically additive over some range, but rarely scalable. A simple compensation can restore often restore linearity by considering a related mixing system.

89. kL 0 L0L0 k*kL 0 knL0knL0 n ddd Scalability in subtractive systems 0<=k<=1

90. L0L0 knL0knL0 n L(nd) = k n L 0 n integer; L(bd) = k b L 0 b arbitrary L(b) = k b L 0 when d = 1; L(b)/L 0 = k b Scalability in subtractive systems T  = t b where T is total transmittance at wavelength, t transmittance of unit thickness and b is thickness 0<=k<=1

91. Linearity in subtractive systems Absorbance A = -log(T ) defn = -log(t b ) = -blog(t ) = -ba  where a =absorbance of unit thickness so absorbance is scalable when thickness b is the control variable By same argument as for scalability, the transmittance of the "sum" of colors T  and S  will be their product and so the absorbance of the sum will be the sum of the absorbances. Thus absorbance as a function of thickness is a linear mixture system

92. Tristimulus Linearity [X mix Y mix Z mix ] = [X 1 Y 1 Z 1 ] + [X 2 Y 2 Z 2 ] c [X Y Z] = [cX cY cZ] This is true because –r( ) g( ) b( ) are the basis of a 3-d linear space (of functions on wavelength) describing lights –Grassman's laws are precisely the linearity of light when described in that space. –[X Y Z] is a linear transformation from this space to R 3

93. Monitor (non)Linearity L 1 (A,B,C) L 2 (A,B,C) L 3 (A,B,C) f 2 (L 1, L 2, L 3 ) ABCABC Linear stage Non-linear stage f 1 (L 1, L 2, L 3 ) f 3 (L 1, L 2, L 3 )

94. Monitor (non)Linearity In = [A,B,C] --> L = [L 1, L 2, L 3 ]--> Out = [O 1 O 2 O 3 ] = [f 1 (L 1, L 2, L 3 ) f 2 (L 1, L 2, L 3 ) f 3 (L 1, L 2, L 3 )] Interesting monitor cases to consider: –In = [dr dg db] where d r, d g, d b are integers 0…255 or numbers 0…1 describing the programming API for red, green, blue channels –Out = [X Y Z] tristimulus coords or monitor intensities in each channel –Typically: f i depends only on L i f i are all the same f i (u) = u  for some  characteristic of the monitor

95. Monitor (non)Linearity Warning: LCD non-linearity is logistic, not exponential but flat panel displays are usually built to mimic CRT because much software is gamma- corrected (with typical  =2.4-2.7) Somewhat related: Most LCD displays are built with analog instead of digital inputs, in order to function as SVGA monitors. This is changing.

96. Monitor (non)Linearity drdgdbdrdgdb RGBRGB a 0 0 0 a 0 0 0 a = + b 0 0 0 b 0 0 0 b where a=1.02/255, b= -.02 (CRT Colorimetry example of Berns, p. 168-169) Non-linearity is f(u)=u ,  = 2.7, same for all output channels. Linearity is diagonal:

97. 050100150200250300 0 20 40 60 80 100 120 R+G+B vs. gray, LCD projector

98. More depth on Gamma Poynton, Gamma and its disguises: The nonlinear mappings of intensity in perception, CRTs, film and video. SMPTE Journal, 1993, 1099-1108

99. Halftoning The problem with ink: it’s opaque Screening: luminance range is accomplished by printing with dots of varying size. Collections of big dots appear dark, small dots appear light. % of area covered gives darkness.

101. Halftoning references A commercial but good set of tutorials Digital Halftoning, by Robert Ulichney, MIT Press, 1987Digital Halftoning Stochastic halftoning

102. Color halftoning Needs screens at different angles to avoid moire moire Needs differential color weighting due to nonlinear visual color response and spatial frequency dependencies.

103. Halftone ink May not always be opaque Three inks can give 2 ^3 =8 distinct colors Visual system gives more since dot size, spacing, yields intensity, gives somewhat additive system Highly nonlinear. See Berns et al. The Spectral Modeling of Large Format Ink Jet PrintersThe Spectral Modeling of Large Format Ink Jet Printers

104. From http://www.matrixcolor.com/

108. 108°162° 90°45°

109. Quantization If too few levels of gray, (e.g. decrease halftone spot size to increase spatial resolution), then boundaries between adjacent gray levels become apparent. This can happen in color halftoning also. See demo at http://www.ctr.columbia.edu/~sfchang/cour se/dip/demos/Quat.html http://www.ctr.columbia.edu/~sfchang/cour se/dip/demos/Quat.html

110. Saturation Distance from white point Adding white desaturates but does not change hue or perceptual brightness. HSB model is approximate representative of this. See PhotoShop

111. Device Independence Calibration to standard space –typically CIE XYZ Coordinate transforms through standard space Gamut mapping

112. Device independence Stone et. al. “Color Gamut Mapping and the Printing of Digital Color Images”, ACM Transactions on Graphics, 7(4) October 1998, pp. 249-292. The following slides refer to their techniques.

113. Device to XYZ Sample gamut in device space on 8x8x8 mesh (7x7x7 = 343 cubes). Measure (or model) device on mesh. Interpolate with trilinear interpolation –for small mesh and reasonable function XYZ=f(device 1, device 2, device 3 ) this approximates interpolating to tangent.

114. XYZ to Device Invert function XYZ=f(device 1, device 2, device 3 ) –hard to do in general if f is ill behaved –At least make f monotonic by throwing out distinct points with same XYZ. e.g. CMY device: –(continued)

115. XYZ to CMY Invert function XYZ=f(c,m,y) –Given XYZ=[x,y,z] want to find CMY=[c,m,y] such that f(CMY)=XYZ –Consider X(c,m,y), Y(c,m,y), Z(c,m,y) –A continuous function on a closed region has max and min on the region boundaries, here the cube vertices. Also, if a continuous function has opposite signs on two boundary points, it is zero somewhere in between.

116. XYZ to CMY –Given X 0, find [c,m,y] such that f(c,m,y) = X 0 –if [c i,m i,y i ] [c j,m j,y j ] are vertices on a given cube, and U=X(c,m,y)- X 0 has opposite sign on them, then it is zero in the cube. Similarly Y, Z. If find such vertices for all of X 0,Y 0,Z 0, then the found cube contains the desired point. (and use interpolation). Doing this recursively will find the desired point if there is one.

117. Gamut Mapping Criteria: –preserve gray axis of original image –maximum luminance contrast –few colors map outside destination gamut –hue, saturation shifts minimized –increase, rather than decrease saturation –do not violate color knowledge, e.g. sky is blue, fruit colors, skin colors

118. Gamut Mapping Special colors and problems –Highlights: this is a luminance issue so is about the gray axis –Colors near black: locus of these colors in image gamut must map into something reasonably similar shape else contrast and saturation is wrong

119. Gamut Mapping Special colors and problems –Highly saturated colors (far from white point): printers often incapable. –Colors on the image gamut boundary occupying large parts of the image. Should map inside target gamut else have to project them all on target boundary.

120. CRT Printer Gamuts

121. Gamut Mapping First try: map black points and fill destination gamut.

122. device gamut image gamut

123. translate B i to B d device gamut image gamut bs (black shift)

124. translate B i to B d scale by csf device gamut image gamut

125. translate B i to B d scale by csf rotate device gamut image gamut

126. Gamut Mapping X d = B d + csf R (X i - B i ) B i = image black, B d = destination black R = rotation matrix csf = contrast scaling factor X i = image color, X d = destination color Problems: Image colors near black outside of destination are especially bad: loss of detail, hue shifts due to quantization error,...

127. shift and scale along destination gray X d = B d + csf R (X i - B i ) + bs (W d - B d )

128. Fig 14a, bs>0, csf small, image gamut maps entirely into printer gamut, but contrast is low. Fig 14b, bs=0, csf large, more contrast, more colors inside printer gamut, but also more outside.

129. Saturation control “Umbrella transformation” [R s G s B s ] = monitor whitepoint [R n G n B n ] new RGB coordinates such that R s + G s + B s = R n + G n + B n and [R n G n B n ] maps inside destination gamut First map R R s +G G s +B B s to R R n +G G n +B B n Then map into printer coordinates Makes minor hue changes, but “relative” colors preserved. Achromatic remain achromatic.

130. Projective Clipping After all, some colors remain outside printer gamut Project these onto the gamut surface: –Try a perpendicular projection to nearest triangular face in printer gamut surface. –If none, find a perpendicular projection to the nearest edge on the surface –If none, use closest vertex

131. Projective Clipping This is the closest point on the surface to the given color Result is continuous projection if gamut is convex, but not else. –Bad: want nearby image colors to be nearby in destination gamut.

132. Projective Clipping Problems –Printer gamuts have worst concavities near black point, giving quantization errors. –Nearest point projection uses Euclidean distance in XYZ space, but that is not perceptually uniform. Try CIELAB? SCIELAB? Keep out of gamut distances small at cost of use of less than full printer gamut use.