1. AR Dance and Engineering in Augmented Reality Presented by: Michael Korostelev

2. the Computer Fusion Laboratory Director: Dr. Li Bai Exchange scholar Dr. Guohua Huang Students: James Ren Michael Korostelev Michael Huang Fengcheng Wu David Osinski CFL Lab

3. Project Showcase `` ````` `````````` Septa Electronic Payment Project 2004 – now ($150K + $1.0M) Dance and Engineering in Augmented Reality 2010 – now ($50K) Wireless Distributed Operating System 2007 – now ($400K) Wind-Solar Hybrid LED Lighting System 2010 - now Sensing Technology for Import Food Inspection 2009 - now Wireless Industrial Process Monitoring 2008 – now ($200K) Project Showcase

4. Art and Engineering Purely creative

5. Art and Engineering Entertainment

6. Art and Engineering Practical and profitable

7. Art and Engineering Educational

8. CFL Lab Artist Visions Film Festival Visual Treasure Hunt Visitors given clues about the town of Lambertville NJ They explore the town to find QR codes Scan QR codes and attach their own message Visitors’ input becomes collaborative story

9. Art and Engineering These projects bring engineers close to artists and act as a bridge between the disciplines Each having a goal to relieve the apprehension laymen have to technology Develop tools for creative people, rather than develop creative works

10. Motivation Create a medium for rich creative works Inspire ideas from artists and engineers alike Enhance education and increase outreach Establish framework for fundamental research

11. Project Goal Enhance visual experience of a live performance with AR Visually enhance storytelling Engage and involve audience Convey complex ideas Reduce production costs Unbind creative limitations

12. What Is AR ? Display Techniques Head Mounted Displays Spatial Displays Handheld Display

13. Dance and Engineering in Augmented Reality Camera/Projector InstallationAugmented Stage High frame rate digital capture High lumen projector with wide focal range PC with accelerated graphics Marker less stationary objects IR led marked moving objects Volumetric objects Dynamic lighting Floor projection

14. 2D/3D Projector-Camera Calibration

15. Homography (perspective transformation) Stereo Calibration

16. Kinect “Hacks”

17. RGBD Sensor Based Calibration Calibrate RGBD sensor Uncalibrated overlayDepth and Image camera separated physically so images do not match Two different approaches to create image overlay - Homography - Stereo Calibration

18. Projector – Sensor Extrinsics Real Scene Sensor Capture Projection

19. Determining Projection Matrix Determining the projection matrix

20. Determining Projection Matrix

21. Displaying the scene

22. Displaying the Scene