Wiimote/Kinect Lab Midterm Update Senior Design December 2011, Group 16 Adviser: Dr. Tom Daniels Brenton Hankins Rick Hanton Harsh Goel Jeff Kramer

Project Goal and Concept Provide software to wrap the Microsoft Kinect SDK functions to provide an interface useful to freshmen in CprE 185 Fix current issues with WiiWrap2 – a program currently used to interpret Wiimote input data in CprE 185

System Design Sample Student Code: int main(int argc, char **argv) { int kr = KINECTE_OK; KINECT *handle; // Kinect handle kr = kinect_global_init(); handle = kinect_init(); //initialize handle if(!handle){ kinect_global_cleanup(); return 0; } //modify rgb values for fun kinect_add_video_modifier(handle, &modify_rgb_values); //modify depth video kinect_add_depth_modifier(handle, &clarify_far_objects); //detect circular heads using agorithm kinect add_depth_observer(handle, &detect_heads); //overlay video feed by user defined image kinect_add_video_modifier(handle, &overlay_heads_with_image);. Student defined callbacks

System Design (contd.) Provides functions to analyze and modify frame data before displaying on the output GUI Receives frames and provides it to the program efficiently after processing Part of Microsoft Research Kinect SDK. Handles drivers and hardware issues. Students write callbacks to process the incoming frames and modifying them if desired

Project Status - WiiWrap Analyzed & solved current software problems Implemented buffer flushing to prevent data loss in piping action Switched IR reporting to raw values from IR sensor Set up memory in WiiWrap to record IR position values between visible instances (in case of temporary loss of signal) Added code for better usability in WiiWrap GUI Fixing user program vulnerability (user program failure would shut down entire GUI) Created Installer Package

Project Status - Kinect Implementing code with SDK Started implementation this semester using the Kinect SDK rather than OpenKinect library, resolved system issues. Observed Depth and Video data streams and made modifications to the frames in an effort to understand student problem solving Implemented Frame processing engine in C++ to receive frames from the Kinect and send them to the C Interface Partly done with the basic C interface to receive and modify frames Modifying open source GUI for displaying output and running student code

Testing and Evaluation We are using similar test plans for both the Wii and Kinect. Initially we will use black box testing methods on individual modules of the program. Testing boundary inputs and random inputs The second phase of testing will be integration testing. Testing the entire system as a whole single unit GUI testing can be done alongside the integration phase of the testing plan. Once the functionality is tested, we will need to do performance testing especially on the GUI. Use Eclipse TPTP to test performance. Test GUI input buffering, load tests, etc.

Pending Issues Need to get GUI implementation completed Need to work on implementing and documenting student Kinect API to provide fuller functionality from the SDK Propose ideas for future CprE 185 labs using Kinect Do testing with current 185 students Document WiiWrap2 functionality in simple terms so it can be provided to middle school and high school teachers Waiting for CSG to provide computer with root or proper hardware/software to test code against 185 lab environment

Project Success Criteria Full Success Installer created and tested for improved version of WiiWrap2 Provide raw numerical Kinect data to the user Provide Kinect SDK interface to the user for the user to modify graphical output in real-time Provide a GUI for graphical output with options for various display outputs (RGB, Depth, Skeleton) Create demonstration labs for 185 Partial Success Installer for WiiWrap2 created and tested Kinect can provide raw output to the user Allow user to modify graphical output in real-time (basic GUI)