1. Depth Analysis With Stereo Cameras
Timo Hohn and Leo Nickerson
Group 5

2. Functionality Motivation
Take stereo images of a scene and calculate the distance objects are away from the two cameras.
This distance will be represented on an output image as a color gradient that is displayed on an external VGA monitor.
Motivation
3D depth analysis for remote robot imagery, where more information of the scene is necessary.
Object detection and avoidance applications for robot navigation where IR systems are not available.

3. Hardware Block Diagram

4. Software Flow Diagram Start (User Initiates Analysis)
Collect JPEGS from Both Cameras
Trim Images
To Overlap
Pixel Recognition
Depth Analysis of Matched Pixels
Update output image
Modify Video Buffer to Include Output Image as Overlay

5. Design Challenges
Interfacing the Cameras` NTSC video in and RS232 communication to the DE2 board
Adapting the camera communication library to work well with the DE2 board Using the onboard TV decoder to collect frame buffers from one of the two cameras
Sending manipulated frames to VGA out.
Application Notes are available from previous projects
Pixel Recognition
Open source libraries from the OpenCV project will be used for matching pixels
May perform image subtraction to reduce number of pixels required to analyze

6. Included FPGA and External Components
Cyclone II Processor (currently economy version)
VGA and TV decoder accessed as Avalon-MM slaves
Use of onboard SRAM modules and the possibility of larger SDRAM storage depending on memory use
GPIO pins (Tx, Rx, and GND pins for each camera, which use 3.3V logic)
Additional +5V power supply for camera modules (either USB wall plug or battery packs)
A single onboard push button for initiating image capture

7. Example C++ Module For Finding Distances
#include "DistFind.h" #include <cmath> DistFind::DistFind(int x_res, int fov_degree, double width){ x_res = x_res; fov_rad = (double)fov_degree*PI/180; rad_per_pixel = fov_rad/x_res; width = width; } double DistFind::find(int x_camera1, int x_camera2){ double offset1 = rad_per_pixel/2 + abs(x_camera1-320)*rad_per_pixel; double offset2 = rad_per_pixel/2 + abs(x_camera2-320)*rad_per_pixel; … …
(x_camera1>320) ? offset1*=-1 : ; (x_camera2<320) ? offset2*=-1 : ; double phi1 = PI/2 + offset1; double phi2 = PI/2 + offset2; double angle_c = PI - phi1 - phi2; double sine_ratio = width/sin(angle_c); double side1 = sine_ratio*sin(phi2); double side2 = sine_ratio*sin(phi1); return (phi1<phi2) ? sin(phi1)*side1 : sin(phi2)*side2; }

8. Unit Testing Integration Testing
Using stored images to test depth and pixel recognition tasks
Sending an image out the VGA port to test that interface
Collect raw data from NTSC video and observe its accuracy
Take raw video input from one camera and route it directly to the VGA port
Starting with a basic, high contrast environment with a single object.
Work our way up to more complex environments involving overlapping objects.

9. Optional Features Additional Features Secondary Features
Performing depth calculations on real time video (additional hardware required)
Automation and optimization of frequent image captures (eliminate the need for user initialization)
Secondary Features
Removal of the NTSC video overlay and only display still images
Simplification of analysis environment if pixel recognition algorithms deem inaccurate

10. Questions?