1. Finger Gesture Recognition through Sweep Sensor Pong C Yuen 1, W W Zou 1, S B Zhang 1, Kelvin K F Wong 2 and Hoson H S Lam 2 1 Department of Computer Science Hong Kong Baptist University 2 World Fair International Ltd

2. Outline Motivations Design Criteria Proposed Method Experimental results Conclusions

3. Motivations Vision-based interface Insert some images using face, expression, body movement… Sensor-based interface http://www.blogcdn.com/www.tuaw.com/media/2008/11/mac-101_-multi-touch-tips.jpg http://www.fitbuff.com/wp-content/uploads/2007/10/wii-fitness.jpg There should be a video about the body movement interface Common objective: natural input to replace traditional physical input devices

4. Motivations While many sensor-based gesture input have been developed, there is no algorithm/system using sweep sensor Why Sweep Sensor? low cost No latency problem (fingerprint recognition) popularity

5. Design Criteria User friendliness easily performed by a user. intuitive and easy to understand. User independent Generic for all users. Robustness diversity of patterns captured. Efficiency Real-time application mobile devices

6. t > t 0 left right No left tick right tick Yes feature vector D > 0.5 left right D > 1.3 left tick right tick D > 1/1.3 D < -0.5 noise reduction envelope enhancement input image direction estimation direction index D = D left /D right 04080120 160 200 0 20 40 60 80 100 120 140 160 04080120 160 200 0 20 40 60 80 100 120 140 160 yiyi i y Characteristics Formulate the noise Proposed Method Characteristics Formulate the noise noise reduction envelope enhancement input image direction estimation direction index D = D left /D right 04080120 160 200 0 20 40 60 80 100 120 140 160 04080120 160 200 0 20 40 60 80 100 120 140 160 yiyi i y t > t 0 left right No left tick right tick Yes feature vector D > 0.5 left right D > 1.3 left tick right tick D > 1/1.3 D < -0.5 Characteristics Formulate the noise noise reduction envelope enhancement input image direction estimation direction index D = D left /D right 04080120 160 200 0 20 40 60 80 100 120 140 160 04080120 160 200 0 20 40 60 80 100 120 140 160 yiyi i y t > t 0 left right No left tick right tick Yes feature vector D > 0.5 left right D > 1.3 left tick right tick D > 1/1.3 D < -0.5

7. Input Image Characteristics Different sensor characteristics Noise level

8. Figure 2. The block diagram of feature extraction Segmentation Owing to different sensor characteristics, the gesture images obtained, even the gesture is the same, will be different Segmentation by estimating the sweeping time noise reduction vertical gradient thresholding horizontal projection

9. Segmentation (cont.) blank part sweeping part noise reduction vertical gradient TH thresholding horizontal projection

10. Feature Extraction Time information t (sweeping time) Finger motion information d (direction) Left and right Left diagonal and right diagonal

11. Feature Extraction (left / right) noise reduction Left Right direction enhancement input image direction estimation direction index D = P left - P right i-th fingerprint texture AB CD

12. Feature Extraction (left tick / right tick) noise reduction envelope enhancement input image direction estimation direction index D = D left /D right 04080120 160 200 0 20 40 60 80 100 120 140 160 04080120 160 200 0 20 40 60 80 100 120 140 160 yiyi i y

13. Classification A very simple rule based on a combination of movement Classification tree (decision tree) left right No left tick right tick Yes feature vector t > t 0 D > 1.3 left tick right tick D > 1/1.3 D > 0.5 left right D < -0.5

14. Designed Gestures Left and RightLeft tick and Right tick

15. Experimental Results 2 testing groups 3 technical users – Engineers, and technical managers, research staff (95.0%) 3 Non-technical users – secretary, clerk (86.87%) Test on different sensors 4 different sensors manufacture at different period of time

16. Experimental Results Evaluation interface There should be a video here

17. Experimental Results Results by 3 non-technical staff with 4 different sensors

18. Experimental Results Integrated application with an image viewer There should be a video here

19. Conclusions

20. Thank You