1. ALFRED THOMPSON MICROSOFT ACADEMIC TEAM HTTP://BIT.LY/ALFREDTH Kinect for FRC 2012

2. Outline 2 Introduction Kinect Hardware Kinect for Windows Skeleton, depth, image and audio data Kinect for FRC 2012 Default FRC gestures Programming your own gestures Additional Resources

3. Kinect Hardware 3 Multi-Array Mic Motorized Tilt IR Projector RGB Camera IR Camera

4. Kinect and FRC 2012  The Kinect has multiple powerful sensors for depth, skeleton, video and audio input  The Kinect is likely intended for robot control via the drivers station for the 2012 FRC game 4

5. Kinect for Windows Architecture 5

6. Skeleton Data

7. Kinect Audio  Four microphone array with hardware-based audio processing – Multichannel echo cancellation (MEC) – Sound position tracking – Other digital signal processing (noise suppression and reduction) 7

8. Kinect Sensor Data 8

9. FIRST Kinect Architecture Driver StationRobot FRC Driver Station Kinect for Windows SDK WPI Kinect Server cRIO WPI Lib (Kinect Enabled) Kinect Joystick Position and Skeleton points 9

10. Kinect for FRC 2012 10

11. Programming Teams using C# can change/modify the WPI Kinect Server on the Driver Station Teams using LabView, Java or C++ can process joystick AND skeleton point data on the cRIO robot controller 11

12. Default FRC Gestures 12 Buttons: 1) Head to the Left 2) Head to the Right 3) Right Leg Out 4) Left Leg Out 5) Right Leg Forward 6) Right Leg Back 7) Left Leg Forward 8) Left Leg Back 9) Kinect Control “Enabled”

13. Control Considerations  Robot control is achieved via skeletal position  Gestures/poses relate to skeletal position and inform which control is desired  Consider your gestures carefully!  Response times will be different with Kinect compared to traditional joysticks 13

14. Using the FRC Gestures 14 Labview KinectStick leftArm(1); KinectStick rightArm(2); leftArm.getY() leftArm.getRawButton(#) C++ KinectStick leftArm; KinectStick rightArm; … leftArm = new KinectStick(1); rightArm = new KinectStick(2); … leftArm.getY() leftArm.getRawButton(#) Java

15. Gesture Programming 15 Teams may modify the Kinect Server code (C#) using Visual C# Express.  Allows access to depth/RGB data  Easier to display feedback on dashboard  Keeps cRIO processing burden low  Gesture examples available from general Kinect development community  Teams may use the raw skeleton data sent to the robot to process their own gestures in their chosen language (LabVIEW, C/C++ or Java)  Language familiarity

16. Getting Help 1. FRC Forums 2. Public Kinect for Windows Forums 3. Escalate to Microsoft via official FIRST contacts 16

17. Demo and Questions Go Teams, go! 17

18. Additional Resources 18 Official Kinect for Windows Website FRC Beta Kinect Forum Kinect SDK Quickstart Video Tutorials Coding4Fun Kinect Toolkit Coding4Fun Kinect Projects Visual C# Express Kinect for Windows SDK Beta Forums Getting Started with Microsoft Kinect for FRC document – released at Kickoff Kinect Server Code Walkthough – released at Kickoff