1. Signal and Data Processing CSC 508 Computer Science & Information Systems Department

2. Motivation As the rapid growth in computer technology continues, computers are being connected to the real world in exciting new ways. Traditional I/O devices (e.g. keyboard, mouse, joystick and printer) are effectively noise-free. That is, the noise they generate is detected and eliminated in the digitization process. Recently, we have seen the incorporation of a new generation of I/O devices and applications into the PC that operate on highly complex (and noisy) signals: voice recognition devices, artificial vision sytems, barcode readers, global positioning systems, proximity and motion detectors, heartbeat “detectors”* * ref: http://www.geocities.com/ResearchTriangle/Facility/1914/DKLmain.html

3. Connecting to the Real World Compared with the digitally sanitized and logically deterministic world of computer programming, the real world is a noisy and unpredictable place. The goal of higher fidelity in our measurement and modeling of the world outside the computer, brings with it the need for an understanding and control of noisy signals. Beyond current and soon-to-be-available PC applications is a rapid rise in the number of special-purpose I/O devices with associated embedded processors in a wide range of intelligent appliances. Until recently, the most advanced electronic sensing devices were available only to the military. With recent cutbacks in government R&D spending, the military is beginning to fall behind the private sector in the development of new IR, millimeter wave, UV and visible surveillance and tracking systems. Remote Sensing (Satellite Surveillance) Forward Looking Infrared (FLIR) Weather and Air Traffic Control Doppler Radars Medical/Industrial Computed Tomography (CT) Acoustic Sensing for Non-Intrusive Engine Diagnostic Systems

4. Scope of this Course A comprehensive study of all areas of signal and data processing would encompass the subjects of several doctoral degrees, including mathematics, physics, electronic engineering and computer science. Just a few of the disciplines closely related to signal/data processing are image processing, pattern recognition, robotics, remote sensing, and telecommunications. Obviously we are not going to cover this subject in detail. The goal of this course is to provide the student, interested in signal and data processing, with a solid foundation in the theory and application of signal/data processing. Upon completion of this course, you should be prepared to continue your study of any field in this discipline in a graduate program or through self-study of the available reference materials.

5. An Overview of the Course We first cover the mathematical (and statistical) background necessary to comprehend the theory and applications presented later. Through examples and demonstrations we study classical techniques in linear and non- linear filtering, Fourier theory, state estimation, pattern recognition, and identification theory. We review the application of new techniques in neural networks, genetic algorithms, rule-based expert systems, chaos theory and wavelets. Along the way, we will gain a practical understanding of how to access and manipulate image and sound files using a high-level programming language (C++/Ada). We will also learn about the interface of the digital computer to external sensing and control devices. Finally we will work with a number of embedded DSPs including the Texas Instruments TMS series of DSPs, the Basic Stamp and the PicStic single board processors.

6. Areas of Signal/Data Processing Semantic Processing - Primarily involves contextual analysis and decision support. It is concerned with the formulation of context for the syntactic elements. Includes functions such as path planning, object identification, vision understanding, and long-term autonomous behaviour. Numeric Processing - The standard Signal Processing Functions including signal conditioning, time-dependent processing and object- dependent processing. Syntactic Processing - Involves the development and maintenance of data object histories. Includes functions such as correlation, tracking, feature extraction and state estimation

7. Homework: A “Simple” Example You are given a single frame (one image) from a CCD imaging device that has a number of dead elements. The dead elements produce light (or dark) pixels in the image. [A similar degradation can occur in dual-scan and active matrix display screens.] Assume that you have program accesss to each pixel in the image as you describe a method to “fix” the bad pixels. i-1,j-1i-1,ji-1,j+1 i,j-1i,ji,j+1 i+1,j-1i+1,ji+1,j+1 Compare each pixel to its 8 neighbors... Hint

8. Consider the Following 1. Develop your own approach for “fixing” the bad pixels and then address each of the issues raised below: a. How does your algorithm detect a bad pixel? b. Under what circumstances will your algorithm change a good pixel by mistake. c. Under what circumstances will your algorithm miss a bad pixel? d. How does your algorithm affect an image with no bad pixels? e. How are the pixels close to a bad pixel affected by your algorithm? 2. From your own experiences with still and video cameras what do you know about the limits of resolution (image sharpness) that help you develop your algorithm? 3. In the functional block diagram below, indicate the source of a good image pixel and the source of a bad pixel. That is, where is each type of pixel value determined? Signal Conditioning Image Processing Storage Sensor Object Optics Focal Plane CCD Computer