1. OPTICAL FLOW The optical flow is a measure of the change in an image from one frame to the next. It is displayed using a vector field where each vector represents the apparent velocity originating at that point. The above example shows a simple demonstration of the concept of optical flow. The left two pictures show two frames; the two images show a circle expanding over time. The right picture shows the optical flow from the image, consisting of a number of vectors expanding outward between the circles. Optical flow is calculated by taking the ratio of the spatial derivative and the time derivative in both the x and y directions and combining these components to form a velocity field: MOTION TRACKING CAMERA GROUP 8 AUTHORS: Nathan Lazarus EE’07 Sharanya Srinivasan EE’07 Lucy Zhang EE’07 ADVISORS: Professor Jan Van der Spiegel Dr. Viktor Gruev Mr. Zheng Yang ABSTRACT In the field of robotics, research has become focused on creating mobile, unsupervised robots to perform various tasks. From the car-sized vehicles of the DARPA Grand Challenge to the Predator drones in Afghanistan and Iraq, sophisticated algorithms have achieved great advances. However, these algorithms become a limitation as the focus changes to the smaller battery- powered robots used in many applications. Large, power hungry microprocessors become impossible. This project is focused on designing a compact low power image sensor chip able to perform on-chip calculations of optical flow. Optical flow, the apparent motion in an image, can be used to control vehicle motion and avoid obstacles. Light intensity is converted to electrical current through an array of photopixels. A series of arithmetic circuits are then used to calculate the spatial and temporal derivatives, as well as take their ratio to calculate the optical flow. SENSOR CHARACTERISTICS Size of Chip: 3mm x 3mm Resolution: 84 by 41 pixels Frame Rate: 30 Hz Transistors: 55932 Input/Output Pins: 68 INTEGRATED CIRCUIT DESIGN FLOW SYSTEM DESIGN The image sensor is the centerpiece of the camera, but is by no means the only component. A complete camera needs microcontrollers, voltage regulators, and digital-to-analog converters (DACs) to provide control signals and voltages for the sensor. It also needs operational amplifiers and analog-to-digital converters (ADCs) to convert the output into a usable signal for the computer. DESIGN AND LAYOUT: Because an integrated chip can contain thousands or even millions of components, sophisticated computer aided design (CAD) tools were used to design, simulate, and layout the sensor. FABRICATION: The design is then sent to a semiconductor foundry for manufacture The sensor used an AMI Semiconductor process capable of feature sizes as small as 0.5 microns. TESTING AND CALIBRATION: The sensor must then be calibrated to obtain a usable image; there are over 30 input signals and voltages, each must be carefully timed or set. Sample and Hold Circuit: Remembers previous frame Photodiode: Converts light intensity to voltage Reset Switch: Resets the photodiode to a reset voltage for the next frame Output Amplifiers: Convert output voltage to electrical current X and Y Control Registers: Allow control of individual pixels Processing Unit: Digitally programmable circuit that can implement a variety of digital filters, including the first derivatives shown here. Division Unit: Absolute value and unsigned division circuits to calculate the final optical flow. Image sensor and lens SX28 microcontrollers MAX5240 four channel DAC MAX118 eight channel ADC Output operational amplifiers PROJECT DEMO April 19, 2007 10:00am, 10:30am, 11:00am 11:30 am, 3:00pm Frame 1 Frame 2 Resultant Optical Flow