“Hand Detection and Segmentation in Cluttered Environments” by Christopher Schwarz and Niels da Vitoria Lobo School of Computer Science, University of Central Florida, Orlando, FL
Hand tracking system Detect and segment straight fingered hands ▪ Any complex background ▪ All reasonable lighting conditions ▪ Single frame analysis Existing systems ▪ Work with silhouettes ▪ Work under normal lighting conditions
Preprocessing Detecting candidate fingers Grouping finger detections into candidate hand detections Post-processing candidate hand groups Generating output
Break down image into lines and curves Search for a pair of nearly parallel lines and a fingertip curve to constitute a candidate finger Run Kanade-Lucas-Tomasi (KLT) Feature Tracker to find locations of textured regions
Combination of Burns Line Finder and Canny Edge Detection Curvefinder software is separate program written by Jan Prokaj
Used in phase 4 to strengthen the detection confidence in finger and hand detections in regions of low texture and weaken those in regions of more complex texture.
Build the initial candidate list Remove duplicates Correct bases Reapply Area Tests
Tip is found correctly, but base does not extend to location where finger meets palm Method of base correction on each finger: Sample gradient magnitude in region around original base point. Move sampling bounding box along median line past the original base in increments of several pixels. Compare strength of data in region to previous. Assume new base when magnitude drastically change.
When a finger is extended through base correction, should the original finger be discarded? Apply length-to-width ratio to original If ratio is in range, keep original and add duplicate detection with “corrected” base.
Form initial groups Score groups for correctness Reinstate candidates Remove false positive hand groups Merge similar hand groups Remove fingers from hand groups Repeat once before continuing to phase 4
To get an estimated palm center: 1. Draw a ray along the medial axis of the finger through the center of its base and beyond 2. Take a point in the ray estimated as the palm center by following it for a distance past the base that is a percentage of the length of the finger (0.6x)
To group candidate fingers into candidate hands: 2 candidate fingers are connected if they meet several criteria, for example: ▪ Similar median brightness values ▪ Estimated palm centers are close (Euclidean distance) Results in cliques of fingers
For a detected hand group Average all of the estimated palm centers for the fingers to get the palm center Estimate palm region as a circle around the center, with a radius proportion to the average length of the detected fingers
2 levels of promotion Promotion 1 ▪ Stronger method of promotion ▪ Candidate must match every finger in the group Promotion 2 ▪ Candidate’s tip-base median ray must come within some distance of group’s palm radius ▪ Candidate must point in same direction as other fingers in group
Correct palm center locations Reapply Remove fingers from hand groups Calculate openness rating of each group Reapply Remove false positive hand groups Remove false positive groups according to the positions of others Reapply Correct palm center locations
Improve location of palm center by looking for internal wedges Curves found in negative space of hand and open fingers Points along perimeter of the palm Wedges used to nudge original palm center in the right direction
Detected hands Pink – palm center Blue – detected fingers Green – Maybe class
Curvefinder written in C by Jan Prokaj KLT written in C by Stan Birchfield of Clemson University Line sketch/Finger finder written in Java by Chris Schwarz Curve sketches generated prior to running Fingerfinder
System is good at detecting palm locations, but not straight fingers. Curvefinder not designed to run on Windows Learning how to run Linux image in VMWare Player
Curved fingers Finding fingers using silhouettes and AdaBoosting Get with Dr. Lobo for info on head detection Learn about AdaBoosting Apply new method for finger detection