1. Microsoft Kinect Jason Wong Pierce Nichols Rick Berggreen Tri Le

2. Overview Background History Kinect development Kinect hardware Kinect features Expanding Kinect beyond video games

3. Kinect Technology Development Technology originally conceived in 2005 by Zeev Zalevsky, Alexander Shpunt, Aviad Maizels and Javier Garcia. Microsoft was not able to gain proprietary hold of the depth technology. PrimeSense now partners with the competitor of the Kinect for PC, ASUS Software design by Rare, a british game company and subsidiary of Microsoft Project Natal > Kinect

4. Kinect Processor Mystery Kinect originally included a processor in its specifications Sept 2010 Microsoft announces that the Kinect will not include the stated processor Released Kinect has a processor – Marvell PXA 168 Theories involve sharing resource demand with xBox, microphone spatial calculations, potential for further innovation

5. Inside the Kinect 1)Microhpone Array 2)IR Emitter 3)Depth Camera 4)Tilt Motor 5)USB Cable 6)Color Camera

6. Inside the Kinect

7. Structured Light Scanning

9. Kinect Light Pattern

10. Kinect Depth Map

11. Precision shape measurement, as of turbine blades Precision measurement of volume and planarity Precision measurement of cutting tool surface and structure Resolution on the order of 10 microns Structured Light Applications

12. Turbine Blade Gauging

13. Almost all non-Kinect structured light cameras are very expensive. The Kinect cuts the cost by not being a precision instrument – Low resolution – approximately 1.3mm @ minimum range. – Not much accuracy required to accurately reconstruct gestures etc. How did the Kinect Get to $150?

14. Features Skeletal Recognition and Tracking Beam Formation to identify the current sound source From microsoft sdk

15. Skeletal Recognition Use input depth map Infer body parts using a decision forest Produce joint position Positions are matched to predefined skeletal structures

16. Skeletal Recognition User captured depth image and synthetic motion capture data for training (object recognition base) 100k poses such that no two poses (max joint) are closer than 5 (cm)

17. Skeletal Recognition Body part / joint selection Correspond to the colors

18. Skeletal Recognition Per-pixel Depth Image extraction Randomized decision forest for each part

19. Skeletal Recognition Training parameters and classification accuracy

20. Joint position proposal Local mode-finding approach based on a mean shift with a weighted Gaussian kernel Each mode is therefore pushed back into the scene by a learned z offset zc to produce final joint position proposal

21. Skeletal Recognition Process repeats for each frame at 30Hz. The skeletal structure is tracked to differentiate different people in frame.

22. Audio Features Linear four-element microphone array Microphone array Improve echo cancellation Noise suppression Identify speaker through beam forming Microphone array characteristic: Noise gain and Directivity Index

23. Beam forming with Audio Array Figure 1. Microphone array directivity pattern in 3D, 1khz Figure 2: Nearly constant beam width in the diapason of 300 to 5kHz the microphone array engine searches for the speaker position and acts as if it points a beam at the current speaker. The higher directivity of the microphone array reduces the amount of captured ambient noises and reverberated waves.

24. Microphone Array Characteristics Microphone array Element s TypeNG, dBNGA, dBDI, dB Linear, small2uni-directional-12.7-6.07.4 Linear, big2uni-directional-12.9-6.77.1 Linear, 4el4uni-directional-13.1-7.610.1 L-shaped4uni-directional-12.9-7.010.2 Linear, 4 el second geometry 4integrated-12.9-7.39.9

25. Advances of Kinect Technology Unique, inexpensive, and compact design has led to new applications involving depth perception, video, and imagery Hobbyists take advantage of the depth camera to create automated moving objects Scientists incorporate motion sensing for hands-free assistance of pictures or remote tools

26. Advances of Kinect Technology Examples utilizing kinect: In Munich, students developed robotic car with Kinect to sent through obstacle courses In UK, team created robot to search through post-earthquake rubble for trapped victims Surgeons in Toronto now view critical patient images and x- rays in the surgery room hands-free instead of leaving room In Oxford, neurologist uses facial recognition and motion sensing to aid blind people with special glasses Professors at UW develop haptic rendering system to experience “remote touching” virtual environment

27. Bibliography http://www.eetimes.com/electronics-news/4210757/Teardown--Kinect-has-processor-after-all http://www.digitalspy.com/gaming/news/a284690/rare-kinect-will-be-our-main-focus.html http://www.wipo.int/patentscope/search/en/WO2007043036 http://www.quepublishing.com/articles/article.aspx?p=1768327&seqNum=3 http://www.wired.com/magazine/2011/06/mf_kinect/all/1 https://www.ee.washington.edu/research/colloquium/slides/2011-2012/slides_20111122_chizeck.pdf http://zapp1.staticworld.net/reviews/graphics/products/uploaded/microsoft_kinect_714802_g3.jpg http://www.kinectxbox360offers.net/wp-content/uploads/2010/06/1-Xbox360_kinect.jpg http://static.myce.com/images_posts/2010/11/210496-kinect-hack_original.jpg http://hackedgadgets.com/wp-content/uploads/2010/11/inside-the-microsoft-kinect_2.jpg http://www.techsmart.co.za/data/articles/Microsoft%20Kinect%20hacking%20continues/kinect%20hack.jpg http://hackedgadgets.com/wp-content/uploads/2010/11/inside-the-microsoft-kinect.jpg http://zapp5.staticworld.net/images/article/2011/06/intelrobot-5181389.jpg http://cdn.ubergizmo.com/photos/2010/12/kinect-drives-car.jpg http://www.fastcompany.com/most-creative- people/multisite_files/mcp/imagecache/mcp2011_slideshow_large/mcp2011_images/alex-kipman-microsoft-i1.jpg http://www.youtube.com/watch?v=T_QLguHvACs http://www.youtube.com/watch?v=HiYvDhVmpFU http://www.youtube.com/watch?v=w8BmgtMKFbY