1. Coding4Fun: Build Fun, Cool, Commercial Applications Using the Kinect for Windows SDK Dan Fernandez Director Microsoft Corporation Brian Peek Senior Technical Evangelist Microsoft Corporation

4. IR Emitter Color Sensor IR Depth Sensor Tilt Motor Microphone Array

5. Horizontal Vertical

7. DEMO

8. DDDDDDDDDDDDDPPP 1514131211109876543210 DepthPlayer Index

9. NearDefault 0.4m/1.3 ft0.8m/2.6 ft

10. 1.3’2.6’9.8’13.1’26.2’.4.8348 Default Mode Near Mode Feet Meters UnknownNormal

11. 1.3’2.6’9.8’13.1’26.2’.4.8348 Default Mode Near Mode Feet Meters UnknownToo NearNormalToo Far

12. DEPTH FORMULAS Depth data short[] allData = new short[depthFrame.PixelDataLength]; Get depth for a point (X:160, Y:120, Width: 320) int index = (width * y) + x; // 38,560 Get X, Y for an index (Index: 5,000, Width:320) int x = index % width; // 200 int y = index / width; //15 Distance Formula short depthPoint = allData[index]; int depth = depthPoint >> DepthImageFrame.PlayerIndexBitmaskWidth; // 2,000 Player Formula int player = depthPoint & DepthImageFrame.PlayerIndexBitmask; // 1

13. DEMO

14. Max of two skeletons per Kinect 6 Player Proposals HandLeft WristLeft ShoulderLeft Head ElbowLeft ShoulderRight HandRight WristRight ElbowRight ShoulderCenter FootLeft AnkleLeft KneeLeft HipLeft HipCenter HipRight FootRight AnkleRight KneeRight Spine

15. Z Axis X Axis Y Axis Joints in meters from camera Negative Positive Negative

16. Each joint has associated tracking state Tracked, Not tracked, or Inferred Inferred - Occluded, clipped, or low confidence joints Use TransformSmoothParameters to smooth joint data to reduce jitter Kinect transform smoothing uses Holt Double Exponential Smoothing - http://en.wikipedia.org/wiki/Holt-Wintershttp://en.wikipedia.org/wiki/Holt-Winters Skeleton.ClippedEdges = FrameEdges.Top;

17. Fatigue Kills GesturesUse Coding4Fun ScaleTo extension method

18. DEMO

19. KINECT AS AN OFFICE MONITOR.

20. KINECT AS AN OFFICE MONITOR.

21. DEMO

22. Rotation in camera space, or amount of rotation from parent bone Player orientation stored in HipCenter, or ShoulderCenter when seated Matrix or Quaternion format for animations

23. http://download.codeplex.com/Download?ProjectName=kinectservice&DownloadId=336928 Kinect data over the network using sockets Color, Depth, Skeleton, and Audio Client libraries for WPF, Windows Phone & WinRT

24. DEMO

25. Real-time tracking of multiple faces 2D Mesh and Points 3D Head Pose Animation Units Z Axis

26. 100 2D Points 87 in picture 13 additional for center of eye, corners of mouth, center of nose, head bounding box

27. Pitch +20 -20 Yaw -45+45 Roll -90 +90

28. Neutral Jaw Lowerer Lip Corner Depressor 0+1 0+1 Closed Sad frown Happy smile Neutral ClosedFully open Six Animation Units Uses Candide3 model

29. DEMO

31. SPEECH RECOGNITION GRAMMAR yes yes please yes yeah yep ok please out._value = "Yes"; var grammar = new Choices(); grammar.Add(“yes please"); grammar.Add(“yes"); grammar.Add(“yeah"); grammar.Add(“yep"); grammar.Add(“ok"); Ensure AutomaticGainControl = false

32. DEMO

33. Admin - controls global game state Robots – Movement, punching, hits, and referee mode User Consoles - player health, time User console’s skeleton data determines movement & punching

34. DEMO

35. AAP205 – Leveraging Natural User Interface Technologies to Deliver Improved User Experience (Tim Huckaby) Project Detroit – By Registration BoxingBots – In Expo Hall

36. Visual Studio Home Page :: http://www.microsoft.com/visualstudio/en-ushttp://www.microsoft.com/visualstudio/en-us Jason Zander’s Blog :: http://blogs.msdn.com/b/jasonz/http://blogs.msdn.com/b/jasonz/ Facebook :: http://www.facebook.com/visualstudiohttp://www.facebook.com/visualstudio Twitter :: http://twitter.com/#!/visualstudiohttp://twitter.com/#!/visualstudio Somasegar’s Blog :: http://blogs.msdn.com/b/somasegar/http://blogs.msdn.com/b/somasegar/

37. Required Slide Complete an evaluation on CommNet and enter to win!

38. Scan the Tag to evaluate this session now on myTechEd Mobile