1. Augmented Reality with.NET casey chesnut brains-N-brawn.com Dallas C# SIG January 2008

2. brains-N-brawn.com Pervasive Computing –Tablet PC (MVP 03) –Compact Framework (MVP 04) –Advanced Web Services (MVP 05) –Media Center (MVP 06) –Speech / Office Communication Server (MVP 07) –Location Based Services –Artificial Intelligence –3D –Image Processing –Augmented Reality

3. Source http://www.brains-N-brawn.com/wpfAugReal/ http://www.brains-N-brawn.com/wpf3dPart Environment : 32-bit Vista, VS 2005,.NET 3.0, Logitech QuickCam Pro Web Cam for Notebooks

4. Audience DirectShow? 3D? WPF? Augmented Reality?

5. Outline Intro to Augmented Reality DirectShow Augmented Reality (AR) Libraries WPF 3D

6. Introduction Augmented Reality (AR) - is the combination of real world data with virtual data.

7. Audience What are some current examples?

8. Current Examples Fighter Pilot HUDs QuickCam video conferencing PS3 Eye of Judgment Sports : Football, Hockey, Nascar Aircraft maintenance ARQuake Car GPS HUD … Star Wars chess!

9. Videos QuickCam Eye of Judgment

10. Steps Acquire real-world data (camera, HMD, mobile devices) Recognize real-world data (computer vision, face detection, fudicial markers, GPS) Determine relative position of real-world marker to input source. Display composite of real-world data and virtual object (monitor, HMD, mobile device, projector).

11. Acquire real-world data OpenCV / SharperCV DirectShow / DirectShow.NET Windows Media Foundation …

12. DirectShow Used by Windows Media Player, Media Center Edition, Windows Media Encoder, Movie Maker, … User for audio/video playback, DVD, TV tuners, web cams, media capture, media transcoding, …

13. DirectShow COM GraphEdt tool Graph, Filter, Pin, Configuration Filters (Source, Transform, Sink) DirectShow.NET (C# wrapper) –http://directshownet.sourceforge.net/http://directshownet.sourceforge.net/ Learning the filters is key Not WPF 3D friendly

14. DirectShow.NET C# wrapper Wraps a majority of DirectShow Many samples available, along with GraphEdt.exe, and MSDN documentation Also has beta wrappers for Windows Media Lib and Windows Media Foundation

15. Demo GraphEdt /dshowPlay /wpfArBasic - capture

16. Recognize real-world data Computer Vision Face tracking AR Libraries (fudicial markers) GPS …

17. AR Libraries ArToolKit ArTag ArToolKitPlus –http://studierstube.icg.tu- graz.ac.at/handheld_ar/artoolkitplus.phphttp://studierstube.icg.tu- graz.ac.at/handheld_ar/artoolkitplus.php Studierstube Tracker Goblin XNA (coming)

18. ArToolKitPlus Optimized for mobile devices C++ Lib GPL, no longer under development –Studierstube Tracker is the latest Not well documented, but ArToolKit is Single, Multi-config, Multi-distinct tracking Markers (ID, BCH, Patterns)

19. Videos ARTag Magic Lens Studierstube Mobile

20. Demo /wpfArBasic – AR tracking /wpfArTest

21. Composite real and virtual data GDI OpenGL / Tao Direct3D XNA WPF 3D …

22. WPF 3D Viewport3D MatrixCamera Light ModelVisual3D / Model3DGroup Material Transform3D HitTest

23. WPF 3D 3D Programming for Windows (Petzold) Windows SDK Samples

24. Demo Simple WPF 3D /wpfArBasic – 3D model /wpfArVideo

25. Future Augmented Reality @ Home (book) –Mostly OpenGL and C++, but at least one C# and Tao sample. Based on ArTag library. Goblin XNA (AR library) –For PC use, not on 360 until XNA is extended …

26. Questions