1 CS6825: Image Formation How are images created. How are images created.

2 Image Formation Images are formed when a SENSOR registers RADIATION that has interacted with PHYSICAL OBJECTS

3 Types of Images Photography: reflected light Photography: reflected light Range images: distance Range images: distance Tomography: tissue density Tomography: tissue density Infrared: heat Infrared: heat We will concentrate on the first type (gray scale and color). (gray scale and color).

4 Digital Images are 2D arrays (matrices) of numbers:

5 Digital images light intensities, or color light intensities, or color distances, or distances, or other physical quantities. other physical quantities. Depending on the type of image, the numbers, pixel values, represent: Color image and 3 fields –red, green,blue Blow up – shows pixels Pixel values represent Depth – white nearer

6 Intensity Images Light coming from the world hits the sensor.

7 Pinhole Camera Pinhole cameras Pinhole cameras Abstract camera model - box with a small hole in it Abstract camera model - box with a small hole in it Pinhole cameras work in practice Pinhole cameras work in practice

8 Distant objects appear smaller Distant objects appear smaller Even though B is larger than C it appears the same size because it is more distant. Even though B is larger than C it appears the same size because it is more distant.

9 Physical parameters involved: Optical parameters of the lens Optical parameters of the lens Characterize the sensorCharacterize the sensor Photometric parameters Photometric parameters Characterize the light reflected off the objectCharacterize the light reflected off the object Geometric parameters Geometric parameters Determine the relative position of the object wrt the sensorDetermine the relative position of the object wrt the sensor

10 Optical Parameters Lens type Lens type

11 Optical Parameters Focal length Focal length Lens Types

12 Optical Parameters Field of view Field of view Characterized by Characterized by Vertical and horizontal angles of viewVertical and horizontal angles of view

13 Optical Parameters Angular apertures Angular apertures Larger aperture (larger diameter of opening) Lets in more light, but, can have Smaller aperture (smaller diameter of opening)

14 Optical Parameters Larger Aperture means: Larger Aperture means: More light comes in (can have faster shutter speed …take quick picture and hence capture faster moving objects).More light comes in (can have faster shutter speed …take quick picture and hence capture faster moving objects). BUT, also means incoming light is uncollimated rays, resulting in a sharp focus only for rays with a certain focal length. This means that a greater aperture results in an image that is sharp around what the lens is focusing on and blurred otherwise. SMALLER DEPTH OF FIELDBUT, also means incoming light is uncollimated rays, resulting in a sharp focus only for rays with a certain focal length. This means that a greater aperture results in an image that is sharp around what the lens is focusing on and blurred otherwise. SMALLER DEPTH OF FIELD For an aperture Shutter speed too low Object near flower are not in focus For this larger aperture

15 Optical Parameters Smaller Aperture means: Smaller Aperture means: Less light comes in (need greater exposure time…not good for fast moving objects…get a blur).Less light comes in (need greater exposure time…not good for fast moving objects…get a blur). BUT, also means incoming light is highly collimated rays, resulting in a sharp focus at the image plane for objects an a more varied distance from the point focused on in the scene. GREATER DEPTH OF FIELDBUT, also means incoming light is highly collimated rays, resulting in a sharp focus at the image plane for objects an a more varied distance from the point focused on in the scene. GREATER DEPTH OF FIELD Objects around flower are in focus with smaller aperture

16 Photometric Parameters Type, intensity, and direction of illumination Type, intensity, and direction of illumination Reflectance properties of the viewed object Reflectance properties of the viewed object Effects of the sensor’s structure on the amount of light reaching the photoreceptors Effects of the sensor’s structure on the amount of light reaching the photoreceptors

17 Geometric Parameters Type of projection Type of projection how we model how light goes from an object into the camera.how we model how light goes from an object into the camera. Position and orientation of the sensor Position and orientation of the sensor Perspective distortions - lens aberations. Perspective distortions - lens aberations.

18 A model –Perspective Projection Based on simple pinhole camera, which is not reality, but works for many applications Based on simple pinhole camera, which is not reality, but works for many applications Math Model

19 Pinhole Camera Model How to calculate the image location p’ from the Object point P. X Y Z f OPp x y Non-linear equationsNon-linear equations Any point on the ray OP has image p !!Any point on the ray OP has image p !! x’ = f (X/Z) y’ = f (Y/Z) ’ ’ ’

20 Perspective Projection Parameters f’ = focal length of lens f’ = focal length of lens O = focal point (like pinhole in camera, center of iris in eye) O = focal point (like pinhole in camera, center of iris in eye) P = point in 3D world (coordinates [x y z]) P = point in 3D world (coordinates [x y z]) P’ = point in 2D image plane (coordinates [x’ y’ z’] z’ is fixed) P’ = point in 2D image plane (coordinates [x’ y’ z’] z’ is fixed)

21 Understanding Perspective Projection Know the following to understand perspective projection: Know the following to understand perspective projection: Points go to Points Points go to Points Lines go to Lines Lines go to Lines Planes go to whole image or Half-planes Planes go to whole image or Half-planes Polygons go to Polygons Polygons go to Polygons Preserves the “perspective” of what you see in a scene. Remember from your grade school – the idea of “Perspective Drawing” it is the same model!

22 How things really work with a Lens

23 The reason for lenses! Without a lens, the light from a object point is NOT focused into a single image point, but, as shown on the top, goes to a ellipse type shape. This is not what we perceive as humans (we have our own lenses). The lens can focus the object point to a point in the image.

24 But….Lenses can have problems Unfortunately, lens are not perfect. Unfortunately, lens are not perfect. There are many models for lens imperfections. Perspective projection is a very SIMPLE model of what actually happens…..consider sesors can have problems to (not uniform reception, etc). There are many models for lens imperfections. Perspective projection is a very SIMPLE model of what actually happens…..consider sesors can have problems to (not uniform reception, etc). Spherical Aberration- point P does not focus to a single point in image.

25 Remember Lens Distance from image plane – focal point. Also, know that when you FOCUS a camera, you are moving the lens increasing and decreasing the focal length ….hence bringing into “focus” (hence appearing as one point in the image) different depth object points in the scene Also, know that when you FOCUS a camera, you are moving the lens increasing and decreasing the focal length ….hence bringing into “focus” (hence appearing as one point in the image) different depth object points in the scene Only center point P2 is focused Perfectly to a point. P1 and P2 are “blurry” not focused to a point …but, Could be if moved lens…focal distance Changes. P1 P2 P3

26 More on Depth of Field Reinforcing last slide…here a different image showing what is in focus and what is not. Reinforcing last slide…here a different image showing what is in focus and what is not. When not focuses on, the point becomes a circle, called circle of confusion. Here only the cat B is focused on perfectly. Both Dog and Rabbit create circles of confusion and will be blurry. When not focuses on, the point becomes a circle, called circle of confusion. Here only the cat B is focused on perfectly. Both Dog and Rabbit create circles of confusion and will be blurry.

27 Conclusion Many kinds of images Many kinds of images Discussed 2D image Formation Discussed 2D image Formation Simple model of how points in image register in the image plane is “Perspective Projection”. Simple model of how points in image register in the image plane is “Perspective Projection”. Discussed Perspective Project Model Discussed Perspective Project Model Described why we need lenses. Described why we need lenses. Discussed what focusing a camera does. Discussed what focusing a camera does.