1. Introduction to Microsoft Kinect Sensor Programming
Presented by:
Mahmoud Awad
Microsoft PhD Scholar

2. What is Kinect Sensor Full Skeleton Tracking Gesture Recognition
Intuitive Interactive Interface
Voice Recognition Capabilities
Affordable
Easy to install

3. What is Kinect Sensor
RGB Camera – A regular video camera that has a resolution of 640x480 at 30 frames per second
3D Depth Sensors – This is a combination of a depth projector (left reticule) and sensor (right reticule) to calculate distance.
Motorized Tilt – The motorized tilt enables you to adjust the camera up or down 27 degrees to ensure that the camera has the best possible view
Multi-Array Microphone – A four-microphone array that is mounted as a single microphone

4. What is Kinect Sensor

5. Installing Kinect Sensor
Download the 32-bit or 64-bit version of the Kinect for Windows SDK at
Visual Studio 2010 (Professional or Express edition)
DirectX
DirectX® SDK - June 2010 or later version
Current runtime for Microsoft DirectX® 9

6. Installing Kinect Sensor
For Speech Recognition:
Microsoft Speech Platform Runtime, version 10.2, we suggest that you install both the 32-bit and 64-bit runtime.
Microsoft Speech Platform - Software Development Kit, version 10.2 – select 32-bit or 64-bit according to your Windows installation
Kinect for Windows Runtime Language Pack, version 0.9

7. Kinect SDK Fundamentals
Start New Windows Presentation Foundation Project
Add a reference to:
Mucrosoft.Research.Kinect
Add “Using” statement
using Microsoft.Research.Kinect.Nui;
using Microsoft.Research.Kinect.Audio;

8. Kinect SDK Fundamentals
Initializing the runtime in the  Window_Loaded event
 if (Runtime.Kinects.Count == 0)
    {
        this.Title = "No Kinect connected";
    }
    else
        //use first Kinect
        nui = Runtime.Kinects[0];
        nui.Initialize(RuntimeOptions.UseColor | RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking);

9. Kinect SDK Fundamentals
We can:
Use more than one kinect but;  can only have one Kinect do skeletal tracking at a time.
Track kinect status changed:
Connected
Disconnected
Not powered
Not ready
error

10. Setting Runtime Options
in the Window_Loaded event, initialize the runtime with the options you want to use. For example, set RuntimeOptions.UseColor to use the RGB camera and Depth Data:
nui.Initialize(RuntimeOptions.UseColor | RuntimeOptions.UseDepth);
The example below sets the runtime to use the color camera, a depth camera, and skeletal tracking:
nui.Initialize(RuntimeOptions.UseColor | RuntimeOptions.UseDepthAndPlayerIndex | RuntimeOptions.UseSkeletalTracking );

11. Uninitializing the Runtime
Remember that when you are done using the Kinect Runtime, you should call the Uninitialize method
nui.Uninitialize();

12. Kinect Events Register Events then open streams:
RGB Camera:  VideoFrameReady
Depth Camera: DepthFrameReady
Tracked Skeleton: SkeletonFrameReady
Open a video or a depth stream.

13. RGB Camera Event
nui.VideoFrameReady += new EventHandler<ImageFrameReadyEventArgs>(nui_VideoFrameReady); //hit tab twice
void nui_VideoFrameReady(object sender, ImageFrameReadyEventArgs e) {
   // event code goes here }

14. video frame ready returned values

15. Converting the byte[] array to an image
PlanarImage imageData = e.ImageFrame.Image;
image1.Source = BitmapSource.Create(imageData.Width, imageData.Height, 96, 96, PixelFormats.Bgr32, null, imageData.Bits, data.Width * imageData.BytesPerPixel);

16. Open Streams Video stream:
nui.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
Depth Stream: 
nui.DepthStream.Open(ImageStreamType.Depth, 2 ,ImageResolution.Resolution320x240,  ImageType.DepthAndPlayerIndex);

17. Calculating Distance There are 2 bytes per pixel.
Use bitshift operator to get the distance for a particular pixel.
private int GetDistanceWithPlayerIndex(byte firstFrame, byte secondFrame) {
  //offset by 3 in first byte to get value after player index
  int distance = (int)(firstFrame >> 3 | secondFrame << 5);
  return distance; }

18. Skeleton Tracking Runtime object: Initialize Runtime: Register Event:
Runtime nui = new Runtime();
Initialize Runtime:
nui.Initialize(RuntimeOptions.UseSkeletalTracking);
Register Event:
nui.SkeletonFrameReady += new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);

19. Skeleton Tracking In event handler get all tracking data:
void nui_SkeletonFrameReady(object sender, SkeletonFrameReadyEventArgs e) {
    SkeletonFrame allSkeletons = e.SkeletonFrame;
    //get the first tracked skeleton
    SkeletonData skeleton = (from s in allSkeletons.Skeletons
                             where s.TrackingState == SkeletonTrackingState.Tracked
                             select s).FirstOrDefault(); }

20. Joint Positions
A Joint position returns X,Y,Z values as explained below
X = Horizontal position measured as the distance, in meters from the Kinect along the X Axis
Y = Vertical position measured as the distance, in meters from the Kinect along the Y Axis
Z = Distance from Kinect measured in meters

21. Getting head position for example:
skeleton.Joints[JointID.Head].Position.X
skeleton.Joints[JointID.Head].Position.Y
Getting Left hand position:
skeleton.Joints[JointID.HandLeft].Position.X
skeleton.Joints[JointID.HandLeft].Position.Y

22. We can scale the values to maximum x and Y values.
Use ScaleTo method from coding4fun kinect toolkit.
Joint HandRight =  skeleton.Joints[JointID.HandRight].ScaleTo(640, 480);

23. Using TransformSmoothing to for less skeletal jitter
Use  TransformSmooth property.
TransformSmoothParameters parameters = new TransformSmoothParameters();
         parameters.Smoothing = 0.7f;
         parameters.Correction = 0.3f;
         parameters.Prediction = 0.4f;
         parameters.JitterRadius = 1.0f;
         parameters.MaxDeviationRadius = 0.5f;
         nui.SkeletonEngine.SmoothParameters = parameters;

24. For more information http://www.microsoft.com/en-us/kinectforwindows/