1. Image Formation

2. Input - Digital Images Intensity Images – encoding of light intensity Range Images – encoding of shape and distance They are both a 2-D array or matrix of numbers These numbers could be 8-bit (most cases), 10-bit, or 12- bit data for black and white images and 24-bit data (Red, Green, and Blue) for color images.

3. Formation Parameters Lens: focal length, field of view, angular apertures, depth of field Photometric: illumination (type, direction, intensity), reflectance (surface properties), and sensor structure (photoreceptors) Geometric: camera position and orientation and distortion. Image data are discrete and intensity scale is quantified

4. Geometric Projection Mapping 3D Scene Points to 2D Image Points Perspective Projection Geometry or pinhole Camera z f y x x y P(X, Y, Z) Image plane Point P in world coordinates Image plane reflected to get positive coordinates. (virtual image) Center / Focus of Projection Optical Axis Image Center p (x, y, z)

5. Camera frame – the x, y, z coordinate system Perspective Projection ? Transformation is not one-to-one Recognizing or reconstructing objects in a 3D scene from one image is an ill-posed problem. What are we trying to do? Recapture information about the 3D original scene that an image depicts.

6. Full-Perspective Camera Fundamental Equations They are nonlinear – magnification ratio (x/X) depends on Z Weak-Perspective (or scaled orthography) Camera Fundamental Equations They are linear – magnification ratio (x/X) is fixed The scene depth is small relative to the average distance from the camera Orthographic Projection is usually unrealistic m = 1

7. Image Digitization Sampling and Quantization: A continuous function f(x,y) is sampled into a matrix with M rows and N columns. The continuous range of the image function is split into K intervals. Sampling Interval Δx, Δy Square-pixel camera Δx = Δyotherwise Sampling function

8. Sampled Function Aliasing: Distortion of image if under sampling. Anti-aliasing: Interpolation

9. Image Sensor Grid Square Grid Hexagonal Grid

10. Quantization K intervals = 2 b b = number of bits 8 bits per pixel are commonly used. 1 bit for binary image 4 and 6 bits low resolution 10 and 12 bits high resolution 24 bits for color images

11. Digital Image Properties Picture elements with finite size Usually are arranged into a rectangular grid 2-D matrix whose elements are integer numbers Euclidean Distance: Computationally Expensive City Block Chessboard Quasi-Euclidean

12. Euclidean : City Block : Chessboard : Quasi-Euclidean : 2.236 3 2 2.414 when otherwise

13. Histogram Contrast local change in brightness Image Quality correlation, mean quadratic difference, mean absolute difference, maximum absolute difference Image Noise random degradation from acquisition, transmission, processing

14. Problems in Digital Images Geometry Distortion: lens imperfection light beams are not bent correctly Intensity Distortion: lens or light imperfection intensity brighter in the center Scattering: beams or radiation bent or dispersed by the medium through which they pass, aerial and satellite images (water vapor) Blooming: imperfect insulation between cells, saturation and spill/leak to neighboring cells, very bright region CCD Variations: imperfect manufacturing, different response to the same light

15. Clipping or Wrap-Around: saturation or loss high order bits loses sensitivity for bright objects, darker than it should be Chromatic Distortion: different wavelengths bent differently same scene spot may show on different pixels sharp edge (step function) becomes blurry (ramp function) Quantization Effects: mapping intensity to one of discrete gray values mixing and rounding problems, spatial quantization effects

17. Other Terms Nominal Resolution: 500×500 pixels for a 10”×10” area Sub-pixel Resolution: interpolation or other algorithms such as sub-pixel edge detection Field of View (FOV): Angular Field of View Depth of Field (DOF): Range of Depth in Focus

18. Image Pre-Processing Intensity Transformations Position-dependent brightness correction Calibration to get e (i, j)

19. Position-independent Brightness Correction Look-up-table (LUT) q = f (p) p q Image Pre-Processing

21. Geometry Transformation (distortion) Line Non-linearity Distortion Panoramic Distortion Skew Distortion Distance Distortion Perspective Distortion

22. Geometry Transformations Pixel Co-ordinates Transformations Intensity Interpolation Image Pre-Processing

23. Pixel Co-ordinates Transformations Maps the coordinates of the input image pixels to the point in the output image. It is a linear transformation. Needs pairs of (x,y) and (x’, y’) to calculate the coefficients. Usually low-order approximating polynomials m=2 or m=3 are used. More points than coefficients are used to provide robustness for the least mean square method (SVD)

24. Points for calibration must be distributed to express the geometry transformation The higher the degree of the approximating polynomial, the more sensitive to the distribution of these points. Bilinear Transform: warping Affine Transform: rotation, translation, scaling, skewing

25. Obtain New Points (x’, y’) Through Transformation or Find the Correspondences on the Input Image The new (x’, y’) are not integers. Interpolation is needed for values on the integer grids Nearest Integer Step-like boundary problem

26. Linear Interpolation (x,y) (l,k) a b Blurry on the edge

27. Bi-cubic Interpolation Use 16 neighboring points Interpolation kernel 0 <= |x| < 1 1 <= |x| < 2 otherwise 0