1. Computer and Robot Vision II Chapter 12 Illumination Presented by: 傅楸善 & 周奕宏 0952 725 532 r97944037@ntu.edu.tw 指導教授 : 傅楸善 博士

2. DC & CV Lab. CSIE NTU Joke

3. DC & CV Lab. CSIE NTU 12.1 Introduction Two key questions in understanding 3D image formation What determines where some point on object will appear on image? Answer: Geometric perspective projection model What determines how bright the image of some surface on object will be? Answer: radiometry, general illumination models, diffuse and specular

5. refraction of light bouncing off a surface patch: basic reflection phenomenon

6. DC & CV Lab. CSIE NTU 12.1 Introduction Image intensity : proportional to scene radiance Scene radiance depends on the amount of light that falls on a surface the fraction of the incident light that is reflected the geometry of light reflection, i.e. viewing direction and illumination directions

7. DC & CV Lab. CSIE NTU 12.1 Introduction Image intensity : incident radiance : bidirectional reflectance function : lens collection : sensor responsivity : sensor gain : sensor offset 61

8. DC & CV Lab. CSIE NTU

9. DC & CV Lab. CSIE NTU Joke

10. DC & CV Lab. CSIE NTU 12.2 Radiometry Is the measurement of the flow and transfer of radiant energy in terms of both the power emitted from or incident upon an area and the power radiated within a small solid angle about a given direction. Is the measurement of optical radiation.

11. DC & CV Lab. CSIE NTU 12.2 Radiometry Irradiance: the amount of light falling on a surface power per unit area of radiant energy falling on a surface measured in units of watts per square meter. Radiance: the amount of light emitted from a surface power per unit foreshortened area emitted into a unit solid angle measured in units of watts per square meter per steradian Radiant intensity: of a point illumination source power per steradian measured in units of watts per steradian may be a function of polar and azimuth angles

13. DC & CV Lab. CSIE NTU 12.2 Radiometry z-axis: along the normal to the surface element at 0 Polar angle: measured from the z-axis (pointing north) Azimuth angle: measured from x-axis (pointing east)

14. DC & CV Lab. CSIE NTU 12.2 Radiometry The solid angle subtended by a surface patch is defined by the cone whose vertex is at the point of radiation and whose axis is the line segment going from the point of radiation to the center of the surface patch.

15. DC & CV Lab. CSIE NTU 12.2 Radiometry size of solid angle: area intercepted by the cone on a unit radius sphere centered at the point of radiation solid angle: measured in steradians total solid angle about a point in space: steradians

16. DC & CV Lab. CSIE NTU

17. DC & CV Lab. CSIE NTU 12.2 Radiometry : surface area : distance from surface area to point of radiation : angle the surface normal makes w.r.t. the cone axis

18. DC & CV Lab. CSIE NTU 12.2 Radiometry surface irradiance : : area of surface patch : constant radiant intensity of point illumination source

19. law of inverse squares: irradiance varies inversely as square of distance from the illuminated surface to source

20. infinitesimal slice on annulus on sphere of radius r, polar angle, azimuth angle slice subtends solid angle, since

21. DC & CV Lab. CSIE NTU

23. DC & CV Lab. CSIE NTU Joke

24. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function The bidirectional reflectance distribution function is the fraction of incident light emitted in one direction when the surface is illuminated from another direction. ratio of the scene radiance to the scene irradiance

25. Θ:Polar angle φ :Azimuth angle

26. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function differential reflectance model: : polar angle between surface normal and lens center : azimuth angle of the sensor : emitting from : incident to : irradiance of the incident light at the illuminated surface : radiance of the reflected light : ratio of the scene radiance to the scene irradiance

27. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function The differential emitted radiance in the direction due to the incident differential irradiance in the direction is equal to the incident differential irradiance times the bidirectional reflectance distribution function.

28. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function For many surfaces the dependence of on the azimuth angles and is only a dependence on their difference except surfaces with oriented microstructure e.g. mineral called tiger’s eye, iridescent feathers of some birds

29. Iridescent Feathers of Some Birds DC & CV Lab. CSIE NTU http://www.birds.cornell.edu/AllAboutBirds/studying/feathers/color/document_view

30. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function An ideal Lambertian surface is one that appears equally bright from all viewing directions and reflects all incident light absorbing none Lambertian surface: perfectly diffusing surface with matte appearance

31. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function reflectivity r: unitless fraction called reflectance factor white writing paper: r = 0.68 white ceilings or yellow paper: r = 0.6 dark brown paper: r = 0.13

32. DC & CV Lab. CSIE NTU white blotting paper: r = 0.8 dark velvet: r = 0.004

33. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function Bidirectional reflectance distribution function for Lambertian surface

34. DC & CV Lab. CSIE NTU : irradiance, : radiance : polar angel, : azimuth angle, : reflectivity

38. DC & CV Lab. CSIE NTU

41. DC & CV Lab. CSIE NTU 12.2.1 Bidirectional Reflectance Function differential relationship for emitted radiance for Lambertian surface Lambertian surface: consistent brightness no matter what viewing direction power radiated into a fixed solid angle: same in any direction

42. DC & CV Lab. CSIE NTU Example 12.1

43. DC & CV Lab. CSIE NTU Joke

44. DC & CV Lab. CSIE NTU 12.2.2 Photometry photometry: study of radiant light energy resulting in physical sensation brightness: attribute of sensation by which observer aware of differences of observed radiant energy radiometry radiant energy photometry luminous energy radiometry power (radiant flux) photometry luminous flux

45. DC & CV Lab. CSIE NTU On Internet Photometry: is the science of measuring visible light in units that are weighted according to the sensitivity of the human eye.

46. DC & CV Lab. CSIE NTU 12.2.2 Photometry lumen: unit of luminous flux luminous intensity: (w.r.t. radiance intensity) luminous flux leaving point source per unit solid angle has units of lumens per steradian candela: one lumen per steradian illuminance: (w.r.t. irradiance) luminous flux per unit area incident upon a surface in units of lumens per square meter one lux: one lumen per square meter foot-candle: one lumen per square foot

47. DC & CV Lab. CSIE NTU 12.2.2 Photometry one foot =0.3048 meter 1 lux = foot - candles = 10.76 foot - candles luminance: ( w.r.t. radiance ) luminous flux per unit solid angle per unit of projected area in units of lumens per square meter per steradian lumen ／ m 2 -sr

48. DC & CV Lab. CSIE NTU Joke

49. DC & CV Lab. CSIE NTU 12.2.3 Torrance-Sparrow Model : diffuse reflection from Lambertian surface facets : specular reflection from mirrorlike surface facets dependent on the view point whereas is not : reflected light from roughened surface consider surfaces:

50. DC & CV Lab. CSIE NTU Torrance-Sparrow model: : proportion of specular reflection depending on surface s=0: diffuse Lambertian surface s=1: specular surface : wavelength of light

52. DC & CV Lab. CSIE NTU : unit surface normal : unit positional vector of the light source : unit positional vector of the sensor

53. DC & CV Lab. CSIE NTU 12.2.4 Lens Collection lens collection: portion of reflected light coming through lens to film : distance between the image plane and the lens : distance between the object and the lens : distance between the lens and the image of the object : diameter of the lens : angle between the ray from the object patch to the lens center

54. da 1 object ａ da 2 image

55. DC & CV Lab. CSIE NTU 12.2.4 Lens Collection irradiance incident on differential area coming from differential area, having radiance, and passing through a lens having aperture area : foreshortened area of aperture stop seen by : distance from to the aperture

56. DC & CV Lab. CSIE NTU 12.2.4 Lens Collection solid angle subtended by aperture stop as seen from : differential radiant power passing through aperture due to

57. DC & CV Lab. CSIE NTU 12.2.4 Lens Collection radiant power passing through aperture from irradiance incident to : (radiant power reaching is )

58. DC & CV Lab. CSIE NTU 12.2.4 Lens Collection assume, then, thus lens magnification is hence, therefore

59. DC & CV Lab. CSIE NTU 12.2.4 Lens Collection since then the lens collection C is given by

60. DC & CV Lab. CSIE NTU Joke

61. DC & CV Lab. CSIE NTU 12.2.5 Image Intensity The image intensity gray level I associated with some small area of the image plane can then be represented as the integral of all light collected at the given pixel position coming from the observed surface patch, modified by sensor gain g and bias b see

62. DC & CV Lab. CSIE NTU 12.2.5 Image Intensity : light wavelength : sensor responsivity to light at wavelength : radiance of observed surface patch : solid angle subtended by the viewing cone of camera for the pixel : distance to the observed patch : power received for the pixel position

63. DC & CV Lab. CSIE NTU Joke

64. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo In photometric stereo there is one camera but K light sources having known intensities and incident vectors to a given surface patch. In photometric stereo the camera sees the surface patch K times, one time when each light source is activated and the remaining ones are deactivated.

65. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo : observed gray levels produced by the model of Lambertian reflectance n: surface normal vector of the surface patch having Lambertian reflectance r: reflectivity of the Lambertian surface reflectance g: sensor gain b: sensor offset

66. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo if camera has been photometrically calibrated, g, b known let and in matrix form

67. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo if surface normal n known then least-squares solution for reflectivity r: if K = 3 a solution for unit surface normal n:

68. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo if K > 3, a least-squares solution:

69. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo if g, b unknown camera must be calibrated as follows: geometric setup with known incident angle of light source to surface normal surfaces of known reflectivities illuminated by known intensity light source

70. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo : known intensity of light source for kth trial : known incident direction of light source for kth trial : known unit length surface normal vector : known reflectivity of surface illuminated for kth trial : observed value from the camera

71. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo let then unknown gain g and offset b satisfy

72. DC & CV Lab. CSIE NTU 12.3 Photometric Stereo this leads to the least-squares solution for

73. DC & CV Lab. CSIE NTU Joke

74. DC & CV Lab. CSIE NTU 12.4 Shape from Shading nonplanar Lambertian surfaces of constant reflectance factor: appear shaded this shading: secondary clue to shape of the observed surface shape from shading: recovers shape of Lambertian surface from image shading

75. DC & CV Lab. CSIE NTU 12.4 Shape from Shading : unit vector of distant point light source direction assume surface viewed by distant camera so perspective projection approximated by orthographic projection surface point position : projected to image position : surface expression

76. DC & CV Lab. CSIE NTU 12.4 Shape from Shading unit vector normal to the surface at :

77. DC & CV Lab. CSIE NTU 12.4 Shape from Shading gray level at, within multiplicative constant Where and : reflectance map

78. DC & CV Lab. CSIE NTU 12.4 Shape from Shading : penalty constant relaxation method: minimizing original error and a smoothness term criterion function to be minimized by choice of p, q

79. DC & CV Lab. CSIE NTU Horn Robot Vision Fig 10.18 a block diagram of Dent de Morcles region in southwestern Switzerland

80. DC & CV Lab. CSIE NTU Horn Robot Vision Fig 10.19 two orthographic shaded view of the same surface caption

82. DC & CV Lab. CSIE NTU 12.4 Shape from Shading uniform brightness if planar surfaces since, constant surfaces with curvature: surfaces with, provide information about surface height first-order Taylor expression for g:

83. DC & CV Lab. CSIE NTU 12.4 Shape from Shading with boundary conditions on, we can solve unknown surface height and partial derivatives,

84. DC & CV Lab. CSIE NTU 12.4.1 Shape from Focus possible to recover shape from the shading profile of object edges basic idea: cameras do not have infinite depth of field The degree to which edges may be defocused is related to how far the 3D edge is away from the depths at which the edges are sharply in focus.

85. DC & CV Lab. CSIE NTU Joke

86. DC & CV Lab. CSIE NTU 12.5 Polarization illumination source characterized by four factors directionality relative to surface normal in bidirectional reflectance intensity: energy coming out from source spectral distribution: function of wavelength λ polarization: time-varying vibration of light energy in certain direction

87. DC & CV Lab. CSIE NTU Examples

88. DC & CV Lab. CSIE NTU 12.5 Polarization polarization: time-varying vibration of the light energy in certain direction linearly polarized: changes direction by every period circularly polarized: phase angle difference of,thus elliptically polarized: phase angle difference of and different amplitude

89. DC & CV Lab. CSIE NTU Mathematical Meaning of Polarization polarization of light mathematically described by using wave theory

90. DC & CV Lab. CSIE NTU Linearly Polarized

91. DC & CV Lab. CSIE NTU Circularly Polarized

92. DC & CV Lab. CSIE NTU Usefulness of Polarization in Machine Vision At Brewster’s angle, the parallel polarized light is totally transmitted and the perpendicularly polarized light is partially transmitted and partially reflected.

93. DC & CV Lab. CSIE NTU Usefulness of Polarization in Machine Vision This effect can be used to remove the specular reflections from the window or metal surfaces by looking through them at Brewster’s angle.

94. DC & CV Lab. CSIE NTU A polarizer can eliminate reflections on non-metallic surfaces.

95. DC & CV Lab. CSIE NTU No Filter With Polarizer With Warm Polarizer http://www.tiffen.com/polarizer_pics.htm

96. DC & CV Lab. CSIE NTU Joke

97. DC & CV Lab. CSIE NTU 12.5.1 Representation of Light Using the Coherency Matrix natural light: completely unpolarized Coherency Matrix: Representation method of polarization

98. DC & CV Lab. CSIE NTU 12.5.2 Representation of Light Intensity The intensity of any light can be represented as a sum of two intensities of two orthogonal polarization components. S-pol: component polarized perpendicularly to the incidence plane P-pol: component polarized parallel to the incidence plane

99. DC & CV Lab. CSIE NTU Joke

100. DC & CV Lab. CSIE NTU 12.6 Fresnel Equation

101. DC & CV Lab. CSIE NTU 12.7 Reflection of Polarized Light ergodic light: time average of the light equivalent to its ensemble average

102. DC & CV Lab. CSIE NTU 12.8 A New Bidirectional Reflectance Function

103. DC & CV Lab. CSIE NTU 12.9 Image Intensity image intensity can be written in terms of illumination parameters sensor parameters bidirectional reflectance function

104. DC & CV Lab. CSIE NTU 12.10 Related Work reflectance models: have been used in computer graphics and image analysis

105. DC & CV Lab. CSIE NTU Joke

106. DC & CV Lab. CSIE NTU 課程網站 http://140.112.31.93 Account: CV2 Password: DCCV Ps. 注意都是大寫

107. DC & CV Lab. CSIE NTU Project due Mar. 7 use correlation to do image matching find to minimize

108. DC & CV Lab. CSIE NTU

109. DC & CV Lab. CSIE NTU

110. DC & CV Lab. CSIE NTU

111. DC & CV Lab. CSIE NTU P.S. 1

112. DC & CV Lab. CSIE NTU P.S. 2