1. E.G.M. PetrakisImage Formation1 Image Formation occurs when a sensor registers radiation. Mathematical models of image formation: 1.Image function model 2.Geometrical model 3.Radiometrical model 4.Color model 5.Spatial Frequency model 6.Digitizing model

2. E.G.M. PetrakisImage Formation2

3. E.G.M. PetrakisImage Formation3 1. Image Function Mathematical representation of a (digital) image. –Relates to digitization: conversion from continuous signal to discrete function Black & White image Color image Multispectral image f = (f 1, f 2, …, f n )

4. E.G.M. PetrakisImage Formation4 2. Geometrical Model Determines where in the image plane the projection of a point will be located. –the projected image is inverted –(x,y,z) is projected on (x’,y’) –f: focal length

5. E.G.M. PetrakisImage Formation5 Avoid inversion by assuming that the image plane is in front of the center of projection –done automatically by cameras or by the human brain Apply Euclidean geometry –x’ = x f /z and y’ = y f/z –depth z is lost !

6. E.G.M. PetrakisImage Formation6 Depth Computation Acquire a pair of images of the same scene using two cameras (or two images by a moving camera) Two identical cameras separated in the x direction by a baseline distance b The image planes are coplanar

7. E.G.M. PetrakisImage Formation7 A point is projected at two different positions on the two camera planes –their displacement is called “disparity”

8. E.G.M. PetrakisImage Formation8 In certain systems (human eyes) the optical axes of the cameras intersect in space –for any angle there is a surface in space corresponding to d = 0. –the disparities may be d = 0, d 0.

9. E.G.M. PetrakisImage Formation9 Epipolar constraint: even if the cameras are in arbitrary positions and orientation the projections lie on the intersection of camera - epipolar planes

10. E.G.M. PetrakisImage Formation10 Correspondence problem: detection of conjugate pairs in stereo images: –for each point in the left image find the corresponding point in the right image –measure the similarity between points –the points to be matched should be distinctly different from their surrounding points –both region and edge features can be used in stereo matching –the epipolar constraint limits the search space for finding conjugate pairs.

11. E.G.M. PetrakisImage Formation11 3. Radiometrical Model Measures the intensity of the reflected light at a point (x’,y’) of the image plane –it is determined by the physics of imaging The proper term of image intensity is image irradiance but –intensity, brightness, gray value are also used Image irradiance is the power per unit area of radiant energy falling into the image plane –Irradiance is incoming energy –Radiance is outgoing energy (from reflecting surface)

12. E.G.M. PetrakisImage Formation12 The irradiance at point (x’,y’) of the image plane depends on the amount of energy radiated by points (x,y,z) in the scene Two factors determine the radiance emitted by a patch of scene surface: 1.The illumination falling on a surface (depends in its position relative to the distribution of light) 2.The fraction of incident illumination reflected by the surface (depends on surface properties e.g., dull, flat, mirror-like etc.) The reflectance of a surface is given by the Bi-directional Reflectance Distribution Function (BRDF)

13. E.G.M. PetrakisImage Formation13 Scene Radiance –Φ: light energy flux –Α: area of source –θ: angle (surface normal & direction of emission) –dω: incremental solid angle

14. E.G.M. PetrakisImage Formation14 Image irradiance Ideally, an imaging device should be calibrated so that the variation in sensitivity as a function of a is removed.

15. E.G.M. PetrakisImage Formation15 4. Color Model Visible light is an electromagnetic wave in the 400nm – 700nm range The light we see is combination of many wavelengths –spectra: the profile below

16. E.G.M. PetrakisImage Formation16 Each neuron on the retina is either a “rod” or a “cone” (rods are not sensitive to color). Cons come in 3 types: red, green, blue –each responds differently to various frequencies of light. Spectral response functions of cones:

17. E.G.M. PetrakisImage Formation17 The color signal to the brain is obtained by adding the responses of the 3 cones –the color signal consists of 3 numbers. –R,G, B sensors filter the scene radiance E(λ). –each sensor has a different spectral response S(λ).

18. E.G.M. PetrakisImage Formation18 CIE primaries: this figure shows the amounts of the 3 primaries needed to match all the wavelengths of the visible spectrum –the negative value indicates that some colors cannot be exactly produced by adding up the 3 primaries.

19. E.G.M. PetrakisImage Formation19 CIE XYZ Based on the CIE primaries –negative values are transformed to positive –chromaticity values x=X/(X+Y+Z), y=Y/(X+Y+Z), z=Z/(X+Y+Z) –x+y+z=1: two values represent all colors

20. E.G.M. PetrakisImage Formation20 Chromaticity Diagram Visible colors: points in the bell Non-visible colors outside the bell Primaries at edges A white point at centre Saturated colors along the radii from edge

21. E.G.M. PetrakisImage Formation21 Color Representation Several methods –Hardware-oriented: defined to properties of devices (TV, printers) that reproduce colors (RGB, CMY etc.) –User-Oriented: based on human perception of colors (HIS, L * u * v etc.) –Colorimetric (CIE), Physiological (CIE XYZ, RGB), Psychological (HIS, L * u * v etc)

22. E.G.M. PetrakisImage Formation22 RGB Color Space The most popular hardware oriented scheme The colors form a unit cube –r = R/(R+G+B) –g = G/(R+G+B) –b = B/(R+G+B) RGB is good for acquisition and display but not for the perception of colors

23. E.G.M. PetrakisImage Formation23 CMY Color Space Cyan, Magenta, Yellow are complements of Red, Green, Blue Obtained by subtracting light from white For color printing Conversion from RGB to CMY –R = 1 – C –G = 1 – M –B = 1 – Y

24. E.G.M. PetrakisImage Formation24 Munsell Color Space Represented in cylindrical coordinates based on –Brightness: vertical axis –Hue: angular displacement –Saturation: cylindrical radius

25. E.G.M. PetrakisImage Formation25 Color Definitions Brightness: intensity of color, average intensity over all wavelengths Hue: is roughly proportional to the average wavelength of the color percept Saturation: amount of white light in color –highly saturated colors have no white –deep red has S=1, pinks have S  0 P = SH+(1–S)W: Think of a color P as an additive mixture W and H where S controls the proportions of W and H

26. E.G.M. PetrakisImage Formation26 HIS Color Space Represented as a double cone –Intensity: the main axis (white at the top, black at the bottom) –Hue: angle around the axis –Saturation: distance from axis –Saturated colors on maximal circles

27. E.G.M. PetrakisImage Formation27 HSV Color Space Similar to HIS –Value –Hue –Saturation –H = undefined for S = 0 –H = 360 – H if B/V > G/V

28. E.G.M. PetrakisImage Formation28 Color Models for Video (YIQ) YIQ is used for color TV broadcasting

29. E.G.M. PetrakisImage Formation29 4. Spatial Frequency Model Describe spatial variations in the frequency domain of the Fourier Transform:

30. E.G.M. PetrakisImage Formation30 f(m,n): linear combination of periodic waveforms –exp{j2π(ux + vy)} –F(u,v): weight factor of frequency u,v –High u,v  image detail (edges, points etc.) –Low u,v  no detail, smooth areas

31. E.G.M. PetrakisImage Formation31 Fourier Transform Pairs

32. E.G.M. PetrakisImage Formation32 Fourier Transform Pairs (2)

33. E.G.M. PetrakisImage Formation33 Sinc(x,y)

34. E.G.M. PetrakisImage Formation34 (a) Original image (c) Smoothed image (b) Edge Enhanced image Fourier Transform of (b) Fourier Transform of (a) Fourier Transform of (c)

35. E.G.M. PetrakisImage Formation35 Convolution Convolution of f and g: Invert g by 180 0, pass pass g over f and compute h on each point of f Theorem:

36. E.G.M. PetrakisImage Formation36 6. Digitizing Model Digitization: Conversion of continuous signals to discrete. –f(x,y)  f(m,n), 0<= m <= M-1,0<=n<=N-1 –f(m,n) = k (intensity value) 0 <= k <= K-1. –f(m,n): samples taken at equal intervals. Perfect sampling: It is possible to reconstruct f(x,y) from f(m,n) –K,m,n must be large enough

37. E.G.M. PetrakisImage Formation37 Image Sampling Multiply f(x,y) by Sampling function

38. E.G.M. PetrakisImage Formation38 One way to reconstruct an image from its samples f(kT) would be to interpolate suitably between the samples Consider one dimensional signals –in the frequency domain –g(x-kT) interpolation function –T: sampling period

39. E.G.M. PetrakisImage Formation39 F(u) for 1D band limited function Non-overlapping copies of F(u) Overlapping copies of F(u) f c : max frequency

40. E.G.M. PetrakisImage Formation40 F(u,v) 2D band-limited function Non-overlapping copies of F(u,v)

41. E.G.M. PetrakisImage Formation41 Select G(w) that isolates F(w) from its samples Whittaker-Kotelnikov-Shannon theorem: f(x) can be reconstructed if the time distance between the samples is at least 1/2f –2f c : sampling rate –If the signal is not band-limited we have aliasing (interference from high frequencies) –Smooth before sampling

42. E.G.M. PetrakisImage Formation42

43. E.G.M. PetrakisImage Formation43

44. E.G.M. PetrakisImage Formation44 KK