1. Wrist Recognition and the Center of the Palm Estimation Based on Depth Camera Zhengwei Yao ; Zhigeng Pan ; Shuchang Xu Virtual Reality and Visualization (ICVRV), 2013 International Conference on 1

2. Outline  Introduction  Related work  Proposed method  Experimental results  Conclusion 2

3. Introduction 3

4.  Problem: Can not separate a hand from a forearm using color and depth information  Solution: Find wrist to recognize hand 4

5. Related Work 5

6.  Hand segmentation and extraction  Color [11,12]  Depth threshold [13,14]  The location of other body parts [15~17]  Wrist  Wear wristband[14]  Palm detection[18] 6

7. Reference [13] D. Uebersax, J. Gall, and M. Van den Bergh, and L. Van Gool, “Realtime sign language letter and word recognition from depth data”. International Conference on Computer Vision Workshops (ICCV Workshops), 2011 [14] Z. Ren, J. Yuan, and Z. Zhang, “Robust hand gesture recognition based on finger- earth mover's distance with a commodity depth camera”. ACM international conference on Multimedia, 2011 [15] T. I. Cerlinca and S. P. Pentiuc, “Robust 3D Hand Detection for Gestures Recognition”. Proc. the 5th International Symposium on Intelligent Distributed Computing, Delft, 2012 [16] M. Van den Bergh and L. Van Gool, “Combining RGB and ToF cameras for real-time 3D hand gesture interaction”. Workshop on Applications of Computer Vision (WACV), Kona, 2011 [17] K. Fujimura and L. Xia, “Sign recognition using depth image streams”. Automatic Face and Gesture Recognition, 2006 [18] U. Lee and J. Tanaka, “Hand Controller: Image Manipulation Interface Using Fingertips and Palm Tracking with Kinect Depth Data”. Proc. of 10th Asia Pacific Conference on Computer Human Interaction(APCHI), Matsue, 2012 7

8. Related Work  Candescent NUI(Natural User Interface) project  Hand and finger tracking  Develop by Stefan Stegmueller, Swiss  Open source: Use the OpenNI framework with the Kinect sensor http://blog.candescent.ch/ http://candescentnui.codeplex.com/ “Finger direction detection” Blue : cluster centroid Green : palm center Red : fingertips Yellow : hand contour Long lines : finger directions ※ “A Robust Method of Detecting Hand Gestures Using Depth Sensors”, Yan Wen; Chuanyan Hu; Guanghui Yu; Changbo Wang, 2012 IEEE International Workshop on HAVE http://www.camdemy.com/media/11513http://www.camdemy.com/media/11513 8

9. Proposed Method 9

10.  Hand segmentation and palm estimation  Wrist recognition  The center of the palm estimation 10

11. Hand Segmentation and Palm Estimation (1/5)  a. Cluster the hand data  K-means clustering algorithm  Specify the depth range: 0.5~0.8m 11

12. Hand Segmentation and Palm Estimation (2/5)  b. Compute the Convex hull of the hands  The Graham scan algorithm 12

13. Hand Segmentation and Palm Estimation (3/5)  c. Detect the hand contours  Moor-Neighbor tracking algorithm 13

14. Hand Segmentation and Palm Estimation (4/5)  d. Detect the fingertips  Find all candidate points that are both on the convex hull and the contour  The distance of P 0 and P’> threshold 14

15. Hand Segmentation and Palm Estimation (5/5)  e. Estimate the center of the palm  The biggest circle inside the hand contour 15

16. Wrist Recognition  Wrist: pit points  Find an obvious pit point in the contour of hand  Create an appropriate to find another wrist point. inscribed rectangle 16

17. Wrist Recognition (1/4)  Step 1: Find candidate lines of wrist  The ends of the candidate line should not be both fingertips.  The distance of the candidate line should not be less than a specific value. 17

18. Wrist Recognition (2/4)  Step 2: Find the corresponding candidate contours whose ends are the ends of the candidate lines. 18

19. Wrist Recognition (3/4)  Step 3: Find one of the wrist points  Calculate the maximum distance between the candidate line and the corresponding candidate contour.  The largest distance from these maximum distances.  The point with the largest distance is one of the wrist points 19

20. Wrist Recognition (4/4)  Step 4: Find another wrist point  Connect this wrist point to each point in the hand contour, and take these connecting lines as the diagonals of rectangles.  If the rectangle is not inside the hand contour, the corresponding point in the contour is not another wrist point.  Find out the point with the shortest rectangle diagonal as another wrist point. 20

21. Wrist Recognition  Candidate lines  Corresponding contour  Find one of the wrist points  Find another wrist point 21

22. Estimating the Center of the Palm (1/4)  Step 1: Select three points from the hand contour  The three points (P 1, P 2, P 3 ) form an acute triangle. 22

23. Estimating the Center of the Palm (2/4)  Step 2: Find circumcenter O j of the triangle  The O j coordinate  The radius of circle : 23

24.  Step 3: Determine the center  Calculate the distances from each point in the hand contour to the center Formula.  Condition A: The number of distance R ji > R j is bigger than the threshold  Condition B: R j >minR (minR=> the minimum radius of the palm )  If A or B is not satisfied, Step 4 Estimating the Center of the Palm (3/4) 24

25.  Step 4: Find another appropriate palm center  One end-point of these two intersectant chords is replaced by point P min  Repeat step 2 to step 4 until the ending condition is true Estimating the Center of the Palm (3/4) 25

26. Proposed Method  Hand segmentation and palm estimation  Wrist recognition  The center of the palm estimation Fingertips detection 26

27. Experimental Results 27

28. Experimental Results  Device: AMD Athlon(tm)Formula Dual Core Processor Formula CPU, 4GB RAM, NVIDIA GeForce 9600GT Graphics card and Window7 32bit OS  Threshold setting  Depth: 0.5-0.8m  Minimum distance of line: 50  Minimum radius of the palm: 33  #hand contour inside circle: 25~50 28

29. Experimental Results Before After 29

30. Experimental Results  Divide into three groups based on the number of the points inside the hand contour. #contour#inside contour a6257085 b4705919 c5057335 d107411205 e135612362 f115513047 g140716716 h147817961 i102418990 30

31. Experimental Results 31

32. Experimental Results  Improved original algorithm: every 8 th point  The new algorithm: proposed method 32

33. Conclusion 33

34. Conclusion  Propose the wrist recognition algorithm to separate the hand from the forearm,  Propose a new algorithm of estimating the center of the palm to reduce the computing time.  Without Kinect SDK 34