1. Capturing, Encoding and Rendering Gestures using the Kinect Adan Patience Supervisor: Mr. Mehrdad Ghaziasgar Co-Supervisor: Mr. Reginald McDonald Dodds Mentor: Mr. Kenzo Abrahams

2. Overview ▪ Recap ▪ Design Decisions and System Changes ▪ Interface ▪ Implementation ▪ Tools Used ▪ Project Plan

3. Recap Introduction ▪ Many countries lack networking infrastructures which support video chatting applications such as Skype. ▪ Develop towards a video chatting application using the Kinect, that could possibly be used extensively by the deaf in the future.

4. Recap Work Flow

5. Recap High Level Design Input Video Frames Capture Locate and Track User Encode Get Joint Coordinates Write Coordinates to File Render Get Frame and Read Coordinates Inert Key Frame Animate

6. Design Decisions and System Changes Previously ▪ No Configuration File ▪ Append Coordinates to tracking file ▪ Used line number in tracking file to determine frame number ▪ Key frame every 24 th set of coordinates ▪ No real-time Currently ▪ Create Configuration file ▪ Overwrite tracking file containing coordinate points ▪ Create new file to store the frame count ▪ Key frame every 10 th set of coordinates ▪ real-time key framing

7. User Interface ▪ Setting foundation for the application ▪ Application not at a mature enough state for a user interface

8. Implementation Input Update Skeleton Capture Process Skeleton Draw Skeleton Encode Get Joint Coordinates and Frame Count Write Coordinates to File and Frame Count Render Configure User and Get Frame Count Get Coordinates Map Coordinates Insert Key Frame Animate

9. Configure User ▪ Assume 3 Positions for the Configuration File – Rest Arms – Arms Up – Spread Arms ▪ Coordinates are stored in CONF.csv – Average of the centre shoulder position is calculated – Minimum and Maximum of both x and y coordinates are determined for both hands. centre shoulder Minimum y-position Maximum x-position Maximum y-position Minimum x-position

10. Map Coordinates Left Wrist Centre Shoulder Right Wrist -6.5 Units Origin 6.5 Units

11. Tools Used ▪ Platform – Windows 7 x86 ▪ Applications – Blender 2.49b – Visual Studio 2013 – Kinect SDK ▪ Languages – Python 2.6 – C++

12. Project Plan Term1 Identify User Requirements, Analyse User Requirements, Literature Survey Become Acquainted with OpenCV and C++ Term2 Identify Objects and Classes in the OOD and analyse the Class’ methods and attributes, Design UI and Choose the Render Application Familiarise myself with the Kinect SDK and API Term3 Create the Capture and Encode Classes leveraging from the Kinect Develop the mapping function. Create UI and finalise application Term4 Test the application against the User Requirements Optimise where possible and iron out bugs