FPGA-based Platform for Real-Time Stereo Vision Sergiy Zhelnakov, Pil Woo (Peter) Chun, Valeri Kirischian Supervisor: Dr. Lev Kirischian Reconfigurable Embedded Systems Lab Ryerson University, Toronto, ON

MotivationMotivation  Areas of application: –Real-time Stereo-Vision Systems –Telematic Systems: Remote Control of Manipulators in Hazardous Areas –Virtual Reality Systems and Simulators –UAV Navigation Systems –Telemedicine –Surveillance / Security Systems

ObjectivesObjectives  Development of the Run-Time Reconfigurable Platform for implementation, testing and real- time verification of algorithms for stereo-vision stereo-image recognition, and visualization of 3D images  Implementation and test of real-time stereo- video processing algorithms (e.g. Edge Detection in moving objects)

SpecificationSpecification  Functional specification: –The system performs: image capture from two color cameras, stereo video visualization with shutter glasses, image processing (edge detection)  Technical specification –Input: video data with  spatial resolution 640 x 480 pixels  frame rate: 30 fps  color, 8-bit resolution –Output: standard SVGA  resolution: 640 x 480 pixels  frame rate: not less than 60 fps Fastest process: 40nS per pixel output

Platform Components and Links Stereo-Image Capturing & Video Pre-processing Module Video Processing Module Multi-Channel Post-processing Video-output Module CRT Monitor LCD Projector Shutter Glasses Run-time Reconfigurable Multi-Stream Video Processor

Implemented Algorithms  Color matching: (Bayer pattern) – at 2x30 frames/sec and 640x480 resolution

Implemented Algorithms  Color matching

Implemented Algorithms  Edge detection (Robert Cross) background 512 pixels 480 pixels ab cd Max (|d-a|, |b-c|),where

Implemented Algorithms  Edge detection (Robert Cross) A2B2C2D2E2F2 …………………………… Z2Y2X2 512 pixels PC camera DATA P : Previous camera data C : Current camera data Temporary Storage a1b1c1d1e1f1 ……………………… z1y1x1 512 pixels Block RAM 2/1 (previous row data) Calculation = C-b1 or P-a1 ……… Block RAM 1/2 (current row data)

Platform Assembly and Implementation Results  Implementation results –Image capture and visualization on the FPGA based Reconfigurable Functional Unit (RFU) (1st stage - XCV50E; 2nd - XC2V1000) –Edge detection algorithm (Robert Cross) implemented

SummarySummary  The Run-Time Reconfigurable Platform was developed for different Stereo-Vision applications.  All components of the Platform were tested by implementation of real-time stereo-image capture, image processing and visualization on stereo- video output display system  Perspectives of the platform development: stereo- panoramic vision system