Visual Information Processing

Human Perception V.S. Machine Perception  Human perception: pictorial information improvement for human interpretation  Machine perception: scene data processing for machine understanding.

Visual Information Processing Why processing visual information: - Better perception for visualization - Medical imaging - Video images (e.g., television commercials to feature films) - Storage and transmission - Application for machine intelligence, robotics, multimedia, graphics and human computer interaction technology Topics in this course: - Image processing fundamentals - Image analysis - Motion image processing (video) - Object recognition - Application with state-of-the-art techniques Why processing visual information: - Better perception for visualization - Medical imaging - Video images (e.g., television commercials to feature films) - Storage and transmission - Application for machine intelligence, robotics, multimedia, graphics and human computer interaction technology Topics in this course: - Image processing fundamentals - Image analysis - Motion image processing (video) - Object recognition - Application with state-of-the-art techniques

Basic Concepts Computer Vision - simulate the human visual system, not only “see” the world, but also “understand” the world (emulate human vision – analysis and understanding) Image Processing - pre-process the image for better “see” or “understand” Pattern Recognition - classify and recognize both image content and some other statistic data. Computer Graphics - create or synthesize a virtual image Artificial Intelligence - emulate human intelligence

Digital Image Processing Development  Digital image processing: - Low-level: Primitive operations (e.g., contrast enhancement, sharpening in Photoshop and Photo-Stacker); Image->image - Mid-level: Image segmentation, classification; image->attributes (e.g., edges, objects). - High-level: Ensemble of recognized objects (vision: make it understood) 1920’s - Digitized newspaper picture transmitted through submarine cable (London New York) - 5 distinct brightness level -> 15 levels  Digital image processing: - Low-level: Primitive operations (e.g., contrast enhancement, sharpening in Photoshop and Photo-Stacker); Image->image - Mid-level: Image segmentation, classification; image->attributes (e.g., edges, objects). - High-level: Ensemble of recognized objects (vision: make it understood) 1920’s - Digitized newspaper picture transmitted through submarine cable (London New York) - 5 distinct brightness level -> 15 levels

Digital Image Processing Development (Cont’d) 1960’s - Images from space probe (distortion correction – image transform) 1970’s - Computerized Tomography (CT) (a ring of detectors collect the x-rays to represent a slice) 1980’s and later - Computer image processing in industry, biomedical area, military recognition, satellite imagery for weather and environment. - Development of signal processing. 1960’s - Images from space probe (distortion correction – image transform) 1970’s - Computerized Tomography (CT) (a ring of detectors collect the x-rays to represent a slice) 1980’s and later - Computer image processing in industry, biomedical area, military recognition, satellite imagery for weather and environment. - Development of signal processing.

Digital Image Processing Development (Cont’d) left: Original image right: Processed image

Image Representation Image (Monochrome image / color image) -- f(x,y) - two-dimensional light intensity function -- (x,y) denote spatial coordinates; -- the value of f at any point (x, y) is proportional to the brightness (or gray level or gray scale) of the image at that point. Example: y x o

Digital Image Representation Digital image -- an image f(x,y) that has been discretized both in the image coordinates and in brightness -- A matrix – the elements of digital array are called pixels (picture elements, image elements, pels) -- In computer programming: 2D array -- Size: width - number of pixels horizontally height – number of pixels vertically Example:

Image Display Computer Hardcopy Image Processing Software Image Sensors (optical to electronic) scene Specialized Image Processing Hardware (digitizer, ALU Arithmetic logic unit) Image Storage Digital Image Processing System

Digital Image Processing Fundamentals Color Image Processing Image Enhancement Knowledge Base Image Acquisition Image Restoration Image Compression Multiresolution Processing Morphological Processing Representation & Description Object Recognition Image Segmentation

Still Image vs. Motion image  Still Image JPEG, JPEG2000 All the fundamental processing Image synthesis  Motion Image Motion analysis and detection Video processing and transmission (H.261, H.263, H.264, MPEG1, 2, 4, 7, 21)  Still Image JPEG, JPEG2000 All the fundamental processing Image synthesis  Motion Image Motion analysis and detection Video processing and transmission (H.261, H.263, H.264, MPEG1, 2, 4, 7, 21)

Illusion

Image Analysis - Computer Vision Human Vision - 70% information from visual perception - 30% information from sound, touch, taste, smell, …… Computer Vision - object detection (edge, region, texture, color,…) - camera calibration - 2D to 3D (shape from shading, shape from texture, shape from motion, …)

Virtual Image - Image Synthesis  Real Image and Virtual Image: Image analysis: real image - using computer to understand the real world Image Synthesis: virtual image - using computer to create a virtual world (computer graphics)  Real Image and Virtual Image: Image analysis: real image - using computer to understand the real world Image Synthesis: virtual image - using computer to create a virtual world (computer graphics)

Evaluation  Subjective and Objective Evaluation: Subjective: No better way to judge the quality of an image than human vision - rating Objective: pixel-by-pixel comparison - mean square errors measurement  Subjective and Objective Evaluation: Subjective: No better way to judge the quality of an image than human vision - rating Objective: pixel-by-pixel comparison - mean square errors measurement

Recent Development 1.Characteristics:  Better quality, Fast processing, Accurate Detection, and More understanding  Architecture: Parallel algorithm  Robotics and Active vision  Face and Gesture Recognition  Document Image Analysis  Texture Analysis  Motion tracking and Analysis  Color image analysis  Image Segmentation and Feature Extraction  3D Reconstruction 1.Characteristics:  Better quality, Fast processing, Accurate Detection, and More understanding  Architecture: Parallel algorithm  Robotics and Active vision  Face and Gesture Recognition  Document Image Analysis  Texture Analysis  Motion tracking and Analysis  Color image analysis  Image Segmentation and Feature Extraction  3D Reconstruction

Relevant areas  Computer Graphics  Image Processing  Multimedia  Human Computer Interaction/Interface

Applications  Biomedical area (CT images)  Military recognition  Satellite imagery for weather and environment  Motion video (MPEG video)  Still image (JPEG)  TV and Film making

Image Processing Programming and tools  MS Windows: Visual C++  Unix, Linux, Irix: C and C++  Intel Image Processing Library.  Image Vision Library  Image format information (BMP, JPEG, TIF,…)  MS Windows: Visual C++  Unix, Linux, Irix: C and C++  Intel Image Processing Library.  Image Vision Library  Image format information (BMP, JPEG, TIF,…)

In this Class  You will learn: Image Processing & Computer Vision: Basics and application  You will do: Programming Assignments, Course Project (proposal, class presentation, project report)  Class attendance is required.  You will learn: Image Processing & Computer Vision: Basics and application  You will do: Programming Assignments, Course Project (proposal, class presentation, project report)  Class attendance is required.

References  Text books: (1) Digital Image Processing (R. Gonzales) (2) Computer Vision (L. G. Shapiro) Journals: IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE Transactions on Image Processing Computer vision, Graphics and Image Processing Pattern Recognition  Conferences: IEEE International Conf. on Computer Vision and Pattern Recognition IEEE International Conference on Computer Vision IEEE International Conference on Pattern Recognition IEEE International Conference on Image Processing