Robot Vision Module Dott. Emanuele Menegatti Intelligent Autonomous Systems Lab University of Padua ITALY Based on course notes of Prof. Bob Fisher Edinburgh University - UK

…a Computer Vision example... Sorting parts on a conveyor belt

... A Robot Vision example... Navigation and Obstable avoidance ? !! Real Time – Real World !!

Physics of Vision

Illumination

The sensors… Humans Eyelid Iris Lens Retina Optical Nerve Robots Shutter Iris Lens CCD TV Cable

The retina… Horizontal cell Bipolar cell Amacrine cell Ganglion cell Optical Nerve Light

Camera Evolution Pinhole Camera /lessonPlans/pinholeCamera/

Camera Evolution The lens Introduced to collects more lights The film Introduced to store the image The CCD Introduced to acquire directly a digital image (higher performances for certain applications)

How to capture colours The original (left) image is split in a beam splitter

Thin Lens terminology Optical Axis Lens Axis Focal point (Secondary) Focal length (in a real camera: Focal length = the distance between the equivalent center of the lens and the image plane)

Ray Tracing We will consider a simple system composed of a single thin lens, there are simple rules we have to follow to ray-trace: 1.Rays travel in straight lines and change directions only when they encounter a discontinuity of the refractive index 2.It is conventional to have the object on the left of the optical system and the image will form on the right 3.All rays emanating from a single point in space must converge on a single point in the image plane (definition of focus) 4.Any ray entering the lens parallel to the axis on one side goes through the focus point on the other side 5.Any ray entering the lens from the focus point on one side emerges parallel to the axis on the other side

How to look at Vision? Low Level Vision: Middle Level Vision: High Level Vision: Image Level Properties Properties of World Properties of Actual Objects –Feature detection –Lightness –Geometry & Shape –Stereo & 2½D sketch –Motion & Optical Flow –Shape from Shading –Object Representation –Object Recognition –Geometric Invariance

Basic Optics

Images File Composed of: Image File Header Self-description of the image Image dimensions Image type Date of creation Program that created the image Image data Some image formats can handle only limited types of images (bynary or monocrome), but the current formats are evolving toward Multimedia contents

Formati di Immagine Due tipi fondamentali di immagini o meglio due modalità grafiche per rappresentare le immagini: Bitmap Limmagine viene descritta dando il colore pixel per pixel Vettoriale Oggetti presenti nellimmagine descritti in termini matematici

Bitmap Caratteristiche: Risoluzione (ppi) Profondità di colore Compressione Lossy (JPEG) Lossless (GIF) Adatto per sfumature di colore

Bitmap (2) GIF (Lossless) Graphics Interchange Format JPEG (Lossy) Joint Photographic Expert Group

Bitmap (3) GIF GIF colori GIF89 Trasparenza Interlacciamento Immagini multiple (gif animate) Created by CompuServe Inc. JPEG Non ha limiti di colore Adatto per foto Una sola immagine per file Header può contenere preview

Immagini Vettoriali Caratteristiche: Dimensione immagine Descrizione matematica Compatta Non adatta per foto o immagini ricche di dettagli Immagine vettoriale Bitmap (FACILE) Bitmap Immagine Vettoriale (DIFFICILE)

PBM (Portable Bit Map) Family of formats: PBM Supports monochrome bitmaps (1 bit per pixel). PGM Supports greyscale images PPM Supports full-color images PNM Supports content-independent manipulations on any of the three formats listed above P3 # magic number # example from the man page 4 4 # cols & rows 15 # maxval

TIFF (Tag Image File Format) Supports multiple images with 1 to 214 bit depth Can be lossy or lossless Very general and very complex Used by scanners Created by Aldus Corp.

PostScript (ps, eps) Store image data using ASCII characters (7- bit ASCII code) Used for graphics displays and printer Newer versions include JPEG compression Used to include graphics or images in a document

MPEG (Motion Picture Expert Group) Stream-oriented encoding for video Contains video, text, graphics Created by an international group of industry and governments Uses Spatial and Temporal Redundancy It is evolving: MPEG-1: 0.25Mbps (audio) 1.25Mbps (video) MPEG-2: 15Mbps good for TV Future version will recognizes objects and generate their images

The Difference Between CCD and CMOS CCD high-quality, low-noise images. CMOS more flexible (every pixel can be read independently) The light sensitivity of a CMOS chip is lower (Many of the photons hitting the chip hit the transistors instead of the photodiode) CCDs consume 100 times more power than an equivalent CMOS sensor. CMOS extremely inexpensive compared to CCD sensors (Chips can be fabricated on any standard silicon production line) CCD sensors produced for a longer period of time, so they are more mature. They tend to have higher quality pixels, and more of them.

Depth of Field... … depends on: Shutter Opening Sensitive Element Focal Length of the lens Distance of the object Region of confusion Hyperfocal distance

Depth of Field

Task: Sort Parts Vision Goal: Describe and Identify Constraints: Flat Shapes Sit flat Different Areas Opacity Serially delivered Camera Synch Silhouette Detector Counter Size Comparator Threshold