1. Creative Coding & the New Kinect
Ben Lower, Kinect Community Evangelism
Carmine Sirignano, Developer Support Engineer

2. 01 | Shifting the Microsoft Culture
Legos vs. Clay

3. 02 | Shifting the Microsoft Culture
Legos versus Clay

4. Creativity = Exploration + Experimentation + Freedom to Fail + Fun
01 | Shifting the Microsoft Culture
Creativity
= Exploration
+ Experimentation
+ Freedom to Fail
+ Fun

5. Kinect Common Bridge & Unity Plug-in
01 | Shifting the Microsoft Culture
Kinect Common Bridge & Unity Plug-in

6. Kinect Overview

7. Welcome to Kinect A new kind of Camera, that can see in 3D
Build 2014
12/29/2017
Welcome to Kinect
A new kind of Camera, that can see in 3D
A traditional video camera
A 4-element microphone array
A powerful SDK which can:
Recognize that a person is in front of the
sensor, and what position their body is in
A software controllable directional
microphone, and speech recognition technology
And lots more
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

8. Build 2014
12/29/2017
What’s New in K4W v2
Single set of cross-platform Kinect APIs (Desktop, Store, Xbox)
Create & publish Kinect apps to the Windows Store
Still have robust support for WPF & Windows desktop applications
Kinect Unity plugin for both desktop & store apps
Kinect Common Bridge for openFrameworks and Cinder
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9. Sensor Features, Sensor Components
Sensor Overview
Sensor Features, Sensor Components

10. Kinect for Windows v2 Sensor Features
Wider field of view (horizontally and vertically) for depth and color
3x depth fidelity
1080p, HD color camera (30fps)
Lighting independent IR (30 fps)
Improved microphones (zero balanced)
25 skeletal joints for six people -> Bio correct skeletal joints (particularly hips/shoulders/spine) plus more points and rotations
Thumb tracking, end of hand tracking, Open/Closed
Improved range -> .5M near, 4.5M far (extended depth to 8M)

11. Sensor Components Depth Sensor Power Light IR Emitters RGB Camera
Image: iFixit,
Power Light
RGB Camera
Microphone Array

12. Connecting the Sensor SDK Browser (Samples & Tools)

13. Color, Infrared, Depth, BodyIndex, Body, Audio
Kinect Data Sources
Color, Infrared, Depth, BodyIndex, Body, Audio

14. Kinect Data Sources Audio Infrared Depth Color BodyIndex Body
Build 2014
12/29/2017
Kinect Data Sources
Infrared
Depth
Color
BodyIndex
Body
Called “Streams” in V1
Audio
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

15. Kinect Evolution

16. Recommended Hardware CPU -> i7 (recommended)
Build 2014
12/29/2017
Recommended Hardware
CPU -> i7 (recommended)
RAM -> 4GB (recommended)
GPU -> DirectX 11 (required)
USB 3.0 -> Intel or Renesas chipsets (required)
OS -> Windows 8.0/8.1 (required)
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

17. Build 2014
12/29/2017
Data Source Details
Physical image sensors, ColorFrameSource, InfraredFrameSource, DepthFrameSource, BodyIndexFrameSource, BodyFrameSource, AudioFrameSource
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

18. Two Physical “Image” Sensors
Build 2014
12/29/2017
Two Physical “Image” Sensors
Color Sensor
1920 x 1080
30 or 15 fps, based on lighting conditions
IR Sensor
512 x 424
30 fps
Different sensor means different pixel dimensions and aspect ratio. Also has wider field of view so there will be color pixels that don’t map to depth pixels.
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

19. ColorFrameSource 1920 x 1080 array of color pixels
Build 2014
12/29/2017
ColorFrameSource
1920 x 1080 array of color pixels
30 or 15 fps, based on lighting conditions
Mirrored image to match other formats
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

20. InfraredFrameSource Pixel Data Ambient light removed
Build 2014
12/29/2017
InfraredFrameSource
Pixel Data
16-bit IR intensity value
Ambient light removed
Not heat – different frequency
People have used this for: night vision, computer vision applications that need consistent illumination like face recognition, retro reflective materials also show up very well
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

21. DepthFrameSource Range: 0.5 - 8 meters Pixel Data
16-bit distance in millimeters from the sensor’s focal plane

22. BodyIndexFrameSource
Build 2014
12/29/2017
BodyIndexFrameSource
Pixel Data
0 to 5: Index of the corresponding body, as tracked by the body source
> 5: No tracked body at that pixel
Could detect a body moved by comparing one frame to the next
Can be used directly for some background removal
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

23. BodyFrameSource Range is 0.5-4.5 meters
Build 2014
12/29/2017
BodyFrameSource
Range is meters
Frame data is a collection of Body objects each with 25 joints
Each joint has position in 3D space and an orientation
Up to six simultaneous bodies
30fps
Hand State on 2 bodies
Lean
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

24. Build 2014 Jumpstart videos: http://aka.ms/k4wv2jumpstart
12/29/2017
Other Features, Resources
Jumpstart videos:
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25. Hand Pointer Gestures and Speech

26. Fusion, Face, HD Face

27. Recording, Playback, and Gesture Recognition

28. Kinect Studio Visual Gesture Builder Custom Gesture Detection Interactions Fusion

29. Thank You! Kinect Resources SDK & Libraries
Events, Sensors, General Info
K4W v2 SDK ->
Kinect Common Bridge ->
Cinder ->
openFrameworks ->
Creative Coding Setup ->
Metrilus Finger Tracking ->
Vancouver (Nov 8) ->
Purchase Sensor ->
General Info & Blog ->
Tutorials, Content, Support Forum
7 Hours of Deep Content ->
VGB Deep Dive ->
Windows Store oF ->
Developer Forums ->
Thank You!
@rickbarraza
We want your feedback and pull requests!