1. Computer Vision: Parallelize or Paralyze Team Purple Threads CSE Capstone 2012 April 2012

2. Abstract Purple ThreadsPurple Threads Project DescriptionProject Description MotivationMotivation System OverviewSystem Overview First StepsFirst Steps Target Drone PlatformTarget Drone Platform Turret SystemTurret System Final StatusFinal Status

3. Purple Threads Top Row: Duc Tran, Aviral Shrivastava(Sponsor), Gabriel Silva Bottom Row: Nicholas Moulin, Craig Hartmann, Anthony Russo, Nadim Hoque

4. Project Description This project is a real world system of robots that visually demonstrates the benefits of parallelization for computer vision applicationsThis project is a real world system of robots that visually demonstrates the benefits of parallelization for computer vision applications

5. Motivation Computer vision systems have a wide variety of applications, but are very processor intensiveComputer vision systems have a wide variety of applications, but are very processor intensive Parallelization allows the implementation of more advanced computer vision techniques by removing the bottleneck on the processorParallelization allows the implementation of more advanced computer vision techniques by removing the bottleneck on the processor

6. System Overview Project consists of two different robotsProject consists of two different robots –Target Drone Platform Detect & track projectiles Avoid projectiles –Turret System Detect & track the target Hit target with foam darts

7. First Steps Requirement ElicitationRequirement Elicitation –High Level –Low Level Hardware/Software SpecificationHardware/Software Specification Risk ManagementRisk Management Project TimelineProject Timeline BudgetBudget Configuring Development EnvironmentConfiguring Development Environment

8. Target Drone Platform HardwareHardware –Traxxas Slash VXL* –ArduPilot Mega w/ IMU Shield –ION Intel Atom Motherboard* –Microsoft Xbox 360 Kinect Sensor* *Other names and brands may be claimed as the property of others

9. Target Drone Platform SoftwareSoftware –EMGU CV 2.3.0 Open CV Wrapper (C#) –ArduRover Code for ArduPilot –Microsoft Robotics Developer Studio 4* –Kinect for Windows SDK v1.0* *Other names and brands may be claimed as the property of others

10. Target Drone Platform ImplementationImplementation –Assemble Target Drone Hardware Components Power System –Setup Wireless Access to Drone –Configure Drone Software Stack –Implemented Platform Services through RDS

11. Target Drone Platform ImplementationImplementation –Obtain RGB & Depth Image Data From Kinect* –Projectile Detection Color Threshold –Projectile Tracking –Collision Avoidance Algorithm *Other names and brands may be claimed as the property of others

12. Target Drone Platform SetbacksSetbacks –Foam dart too small to track accurately –Hardware too heavy for original shocks –Depth frame coordinates are offset from RGB frame coordinates with no translation function –Dr. Shrivastava crashed the car into a tree!

13. Turret System HardwareHardware –USB Missile Turret –Microsoft Xbox360 Kinect Sensor* –Arduino Uno –Servos SoftwareSoftware –OpenCV 2.3.1 –Libfreenect –Ubuntu 10.04* *Other names and brands may be claimed as the property of others

14. Turret System ImplementationImplementation –Obtain RGB/Depth Images from Kinect* Sensor –Object Detection Color Threshold Cascade Classification –C USB Turret Driver –Program Arduino to Control Servos –C NetDuino Driver *Other names and brands may be claimed as the property of others

15. Turret System ImplementationImplementation –Equations Created to Track Target Depth Disparity to Depth(ft) –d_feet = (disparity-824.8)/15.2 –d_feet = (0.1236*tan((disparity/2842.5)+1.1863))*3.2808399 Pixels per Foot depending on Depth –Pixels/Foot = -55.91836*log(d_feet)+177.49225974457

16. Turret System SetbacksSetbacks –Original turret configuration not precise enough –Unable to implement libfreenect driver on PS3* –Turret caught on fire –Turret mounting unstable *Other names and brands may be claimed as the property of others

17. Turret System Turret Tracking and Hitting a Moving CarTurret Tracking and Hitting a Moving Car http://youtu.be/r7ccvPGcY8Uhttp://youtu.be/r7ccvPGcY8Uhttp://youtu.be/r7ccvPGcY8U

18. Turret System Effect of Parallelization on Turret SystemEffect of Parallelization on Turret System http://youtu.be/D5yTRKgOykYhttp://youtu.be/D5yTRKgOykYhttp://youtu.be/D5yTRKgOykY

19. Final Status

20. Questions ?

21. Backup Slides

22. Turret System ImplementationImplementation –Equations To Track Target Horizontal Rotation Vertical Rotation