1. Capturing Your Audience with Kinect
4/11/2017
Capturing Your Audience with Kinect
Featuring: Speech Bubbles for Kinect v2
By Shahed Chowdhuri
Sr. Technical Evangelist
Title Page: Capturing Your Audience with Kinect Featuring: Speech Bubbles for Kinect v2
By Shahed Chowdhuri
Sr. Technical Evangelist
Blog: WakeUpAndCode.com
WakeUpAndCode.com
@shahedC
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

2. Agenda Introduction > Theory > Code > Demo Next Steps Agenda

3. Tools & Technologies Tools & Technologies Visual Studio 2013 Windows 8
Microsoft .NET and Visual C#
Kinect v2
JavaScript

4. Introduction 4/11/2017 Introduction
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

5. Capturing Your Audience…

6. Capturing Your Audience… with Kinect!
Source:

7. What is Kinect? Depth Sensor Power Light RGB Camera IR Emitters
Microphone Array
RGB Camera
IR Emitters

8. 4/11/2017
What’s New in v2?
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

9. What’s New in v2? 1080p color camera Improved depth Better infrared
More joints

10. Color Camera: What’s New
30 fps
1920 x 1080
3x depth fidelity
Wider view

11. Depth Sensor: What’s New
Lighting-independent
0.5 to 8 meters
No more short range mode

12. InfraRed: What’s New? Better infrared Ambient light removed
30fps

13. Body Tracking 25 skeletal joints Up to 6 people
Hand states on 2 bodies
open/closed, lasso
Joint rotation

14. Reading Stream/Frame Data
4/11/2017
Reading Stream/Frame Data
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

15. How Do You Get the Data?
Sensor
Source 
Frame 
Data 

16. Detect the Sensor
Detect the sensor

17. Capture the Source
Open a “Reader” to capture the Source

18. Acquire a Frame
Acquire a Color Frame

19. Process the Data
Draw Pixel Data

20. Mapping Coordinates 4/11/2017
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

21. Acquiring Body Frames to Draw Joints
Acquire Body Frame
Draw Joints

22. Using the Coordinate Mapper
Get 2D point
Draw Comic Bubble in 2D coordinates

23. Start Drawing!
offset from joint
on the joints

24. 4/11/2017
Facial Expressions
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

25. Possible Facial Expressions

26. Capturing Frame Results
acquire frame

27. Determine Results
Reference: Tom Kerkhove’s “First Look at Expressions”:

28. Use Case: Speech Bubbles!
4/11/2017
Use Case: Speech Bubbles!
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

29. Speech Bubbles for Kinect v2
Demo

30. 4/11/2017
What’s Next?
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

31. What’s Next?
Windows Store App, Unity Pro plugin, Xbox One

32. Technology Choices Windows C# & XAML/WPF C++ & DirectX
HTML5/JS (store only)
Unity C#
JavaScript

33. Resources/Links:
Get Kinect v2 for Windows (requires USB3 port and Windows 8.1):
Download Visual Studio 2013 Community Edition:
Download free Kinect for Windows SDK:
Resources/Links:
Get Kinect v2 for Windows (requires USB3 port and Windows 8.1):
us/kinectforwindows/purchas e/
Download Visual Studio Community Edition:
Download free Kinect for Windows SDK:
us/kinectforwindows/develop /

34. Resources/Links: (continued)
Get sample code from Kinect MVP Tom Kerkhove:
Get the code for Speech Bubbles:
Watch MVA Video Tutorial, by Ben Lower and Rob Relyea:
Resources/Links: (continued)
Get sample code from Kinect MVP Tom Kerkhove:
orWindows/G2KBasicOvervie w
Get the code for Speech Bubbles:
peechBubblesForKinect
Watch MVA Video Tutorial, by Ben Lower and Rob Relyea:

35. Contact
Contact
Microsoft
Personal
Dev Blog: WakeUpAndCode.com 