Presentation is loading. Please wait.

Presentation is loading. Please wait.

Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors Spring Semester, 2010 14 th week, Applications.

Similar presentations


Presentation on theme: "Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors Spring Semester, 2010 14 th week, Applications."— Presentation transcript:

1 Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors Spring Semester, 2010 14 th week, Applications

2 Outline 4 Motivation 4 Hand Gesture Recognition 4 Virtual Game Control 4 Experiments 4 Summary

3 Hand Gesture-based HCI

4 Sensing Techniques 4 Vision-based + Easily track hand gestures - Sensitive to user’s circumstances 4 Movement-based –Glove: + Achieve good performance / - wear a cumbersome glove (hinders the convenience and naturalness of HCI) –ACC: Easy to wear and recognize 4 EMG-based –Hands-free application yet less accurate

5 Motivation 4 Each sensing technique has its own advances and capabilities  multiple sensor fusion 4 ACC-based gesture control –Well suited to distinguish noticeable, larger scale gestures 4 EMG-based gesture control –Contain rich information about hand gestures of various size scales 4 We considered the complementary features of ACC (recognition) and EMG (segmentation + recognition)

6 Hand Gesture Recognition 4 Proposed system 4 Segmentation 4 Feature extraction 4 Recognition with HMM

7 Data Segmentation 4 As hand movement switches from one gesture to another one, the corresponding muscles relax for a while 4 Onset threshold –Determine active segments 4 Offset threshold –Prevent the fragmentation Hand Gesture Recognition

8 Feature Extraction: EMG 4 Frame-based extraction 4 Every EMG channel is filtered by Hamming window –Minimize the signal discontinuities 4 4×n dimensional feature vectors –3 rd order auto-regressive coefficients –Mean absolute value Hand Gesture Recognition Window size = 250ms

9 Feature Extraction: ACC 4 Normalize the variations in the scale and speed of gesture –Scale the amplitude of the data –Extrapolate the active segment to 32 points 4 3×32 dimensional feature vectors Hand Gesture Recognition

10 HMM for Recognition 4 Multi-stream HMMs 4 Weight factors 4 Recognition result Hand Gesture Recognition

11 Virtual Game Control

12 Experiments: Setup 4 Delsys Myomonitor IV sensor system –Four-channel EMG + a 3D-ACC 4 Five subjects 4 Utilize LR HMMs with five states –Built EMG and ACC HMMs independently (weight = 0.5) –Training set is collected by 10 repetitions

13 Results Experiments

14 Summary 4 Hand gesture recognition can be utilized in natural interaction between human and computers 4 EMG + ACC to achieve real-time hand gesture recognition  Virtual Rubik’s Cube game


Download ppt "Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors Spring Semester, 2010 14 th week, Applications."

Similar presentations


Ads by Google