RoshamRobo Alexander Ciccone – EEL4665 Spring 2014 Oral Report 2 – Special Sensor Image Credit – Author: Enzoklop URL:

Special Sensor Laptop (Windows or Ubuntu environment) running OpenCV program to interpret images sent from an IP camera (Foscam FI8910W) via a wireless router. RoshamRobo must: Use sensors and IP camera to find and position itself in front of a target human Display countdown and game moves via LCD screen at appropriate times The software and special sensor must: Detect an outstretched hand of varying colors (foreground & background) Differentiate amongst Rock, Paper, and Scissors hand gestures during play Images: s-ln.in, wikipedia, amazon.com

Images courtesy of sparkfun, adafruit, amazon.com Special Sensor Block Diagram

Inspiration Computer vision algorithms are exciting! OpenCV algorithms able to detect & track faces, people, hands, images and many other objects Images: bassemdy.wordpress.com,

Initial Stages Install and troubleshoot installation of OpenCV Compile and study resources on hand gesture recognition and OpenCV applications The following resources were extremely helpful: anikettatipamula.blogspot.ro blogs.ugidotnet.org/WetBlog

Initial Stages Subtract background from image while preserving foreground (hand) Find Convex Hull of hand region (the convex set enclosing the hand)

Convex Hull – Experimental Results

Finding Convexity Defects Find, mark, and count (convexity) defects of the convex hull

Counting Defects & Fingers

Identifying Scissors & Rock Gesture

Further Experimentation