1. Gesture Control interface
Group 2 project

2. Who are we? Rajan Jassal Eric Smith Chris Chmilar
Current lead in Linux side USB communication
Future lead in microcontroller programming
Eric Smith
Current lead in DE2 side USB communication
Future lead in IMU (Inertial Measurement Unit) operations
Chris Chmilar
Current lead in wireless communication
Future lead of gesture recognition

3. The project
A glove for executing navigational operations in a Linux environment by detecting a user’s movements
Gestures can be used to scroll vertically in documents, and switch between windows or tabs.
Touching any finger to the thumb will allow additional macros to be used or change the working mode of the glove.

4. Motivation
Convenience Provide a way to do the little convenient things with your computer intuitively and from a distance: say you’re chopping onions and your computer is on the other side of the counter. Using this device, with just a wave of your hand, you can scroll down to the next step in your recipe. Innovation To explore different ways to wirelessly interact with a computer. Gesture control has been in the mind of innovators for some time, however a useful product has yet to be realized.

5. Top level design (module diagram)
Glove (IMU)
DE2 (Gesture Recognition)
Computer (Custom Driver)

6. Glove components
9-degree of freedom inertial measurement unit (IMU) for tracking motions
Metal contacts for detecting interaction between fingers
Wireless transmitter for sending data to microcontroller
Microprocessor for managing other components

7. De2 and FPGA components
Xbee wireless receiver or receiving data from glove
USB controller (ISP1362) for communication between board and computer

8. Computer and driver
Custom driver for Linux to interpret data from the DE2 board
Performing actions within the OS based on data received

9. Block diagram

10. Dataflow diagram

11. Gestures
Data gathered is interpreted by a program run on the DE2 and compared against hard coded gesture templates. Each template will represent a fairly broad set of movement inputs so as to not be too difficult to use. These gestures correspond to signals to be sent over the USB connection that the custom driver will recognise as mouse or keyboard actions. We may add some learning algorithms to narrow the template boundaries to improve user specific performance

12. Challenges (so far)
Numerous components makeup the project so finding parts that interface correctly took a lot of time and research
Drivers are quite complicated for the functionality we are after and so are taking a lot of research

13. Design calculations
Maximum actions taken per second: Based on a trial done on a touchscreen for number of actions when scrolling rapidly on a screen. 65 actions / 30 seconds = Nyquist rate = 2*2.166 = 4.33 actions / second. ZigBee serial data rates set to baud = 4800 bytes/s. Maximum packet size possible is 1.07 KB

14. Code Example (Pseudocode)
gesture gestureRecognition (packet)
data = unpackData(packet);
gestureSequence = convertToGestureSequence(data);
gesture = matchTemplate(templateList, gestureSequence);
sendToComputer(gesture, USB_CORE_ADDR);
return gesture

15. Test plan
All components will be tested separately and after being integrated
Wireless Communication:
Use indicators to see if transmission is received (ex. LEDs)
IMU:
Attach to a board directly to observe the types of data received
USB Communication:
Pre-set hardcoded commands to be interpreted by the driver
Integrated Testing

16. Optional features Features to add: Features to remove:
Other modes of operation, such as a media mode to play/pause and skip multimedia
A mouse-like mode to add cursor functionality
Features to remove:
The metal contact (“touch”) operation

17. Questions?