1. ECE532 Final Project Demo Disparity Map Generation on a FPGA Using Stereoscopic Cameras ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting

2. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Depth Detection Depth information applied in automotive collision avoidance, XBOX Kinect, robot vision Input stereoscopic image pair compared to produce disparity map Disparity map contains depth information at each pixel location

3. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map to Depth Disparity value is (a – b) Given: f – focal length d – camera distance Depth = z+f = fd/(a-b)

4. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map to Depth Goal: –Produce a disparity map for depth sensing using stereoscopic cameras –Implement in hardware: Adaptive Window Sizing Sum of Absolute Differences Pixel Matching Disparity Map Generation Output Disparity Map Filtering

5. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map Algorithm Variation Window Sizing Areas of less pixel variation receives larger window Areas of more pixel variations receives smaller window Window size information to be used for S.A.D phase Variation Window Sizing S.A.D Pixel Matching Disparity Map Generation Disparity Map Filter

6. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map Algorithm Sum of Absolute Differences Simple cost function for pixel matching between reference and search frames Other cost functions such as sum of square differences Variation Window Sizing S.A.D Pixel Matching Disparity Map Generation Disparity Map Filter

7. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map Algorithm Disparity Map Generation Determines disparity value, pixel coordinate difference of matching pixels in reference and search frames Stores in memory as disparity map Variation Window Sizing S.A.D Pixel Matching Disparity Map Generation Disparity Map Filter

8. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map Algorithm Disparity Map Filter A filter can be applied to remove outliers Variation Window Sizing S.A.D Pixel Matching Disparity Map Generation Disparity Map Filter

9. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Disparity Map Algorithm

11. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Final Hardware System Video In IP Disparity Map Frontend Disparity Map Backend HDMI Out IP MicroBlaze

12. Spartan 6 FPGA disparity_map_frontend asynch fifo (reference image) asynch fifo (search image) pixel conversion BRAM disparity calculation block disparity_map_backend async fifo (disparity as grayscale pixel) address calculation for AXI burst write Master Burst Read Interface Microblaze softprocessor AXI Interconnect Memory Controller Master Burst Write Interface HDMI out IP video in IP VmodCam Reference IP async fifo (CAM A input) async fifo (CAM B input) Master Burst Write Interface DDR2 SDRAM VModCam Stereoscopic Camera Monitor Legend Custom IP Existing IP Peripheral Hardware

13. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Design Process MATLAB implementation MicroBlaze C Program Hardware modules divided among team members

14. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Results MATLAB produced disparity map

15. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Results Hardware produced disparity map

16. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Performance Comparison System Frame Size Time MATLAB 100x200~5 minutes (>1hr for 640x480) MicroBlaze C Program 640x480~20 minutes Hardware System 640x4802 - 8.5 seconds

17. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting What We Learned… Simulation is faster than synthesis How to use EDK/SDK Tools and ModelSim Communicating about interfaces between blocks is key

18. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Demo!

19. ECE532 Final Project Demo Team 3 – Alim, Muhammad, Yu Ting Thank You! Questions/Comments