© 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of 1-1 HCI Human Computer Interaction Week 10

1-2 © 2010 Pearson Addison-Wesley. All rights reserved. What Is Augmented Reality (AR)? A combination of –a real scene viewed by a user and –a virtual scene generated by a computer that augments the scene with additional information. –ARToolkit demo movie –T-immersion 2004 video

1-3 © 2010 Pearson Addison-Wesley. All rights reserved. Augmented Reality vs. Virtual Reality Augmented Reality System augments the real world scene User maintains a sense of presence in real world Needs a mechanism to combine virtual and real worlds Hard to register real and virtual Virtual Reality Totally immersive environment Senses are under control of system Need a mechanism to feed virtual world to user Hard to make VR world interesting

1-4 © 2010 Pearson Addison-Wesley. All rights reserved. Milgram’s Reality-Virtuality Continuum Mixed Reality (MR) Real Environment Virtual Environment Augmented Reality (AR ) Augmented Virtuality (AV) Milgram coined the term “Augmented Virtuality” to identify systems which are mostly synthetic with some real world imagery added such as texture mapping video onto virtual objects.

1-5 © 2010 Pearson Addison-Wesley. All rights reserved. Combining the Real and Virtual Worlds We need: Precise models Locations and optical properties of the viewer (or camera) and the display Calibration of all devices To combine all local coordinate systems centered on the devices and the objects in the scene in a global coordinate system

1-6 © 2010 Pearson Addison-Wesley. All rights reserved. Combining the Real and Virtual Worlds (cont) Register models of all 3D objects of interest with their counterparts in the scene Track the objects over time when the user moves and interacts with the scene

1-7 © 2010 Pearson Addison-Wesley. All rights reserved. Realistic Merging Requires: Objects to behave in physically plausible manners when manipulated Occlusion Collision detection Shadows

1-8 © 2010 Pearson Addison-Wesley. All rights reserved. Research Activities Develop methods to register the two distinct sets (real, virtual) of images and keep them registered in real-time –This often reduces to finding the position of a camera relative to some fiducial markers Develop new display technologies for merging the two images

1-9 © 2010 Pearson Addison-Wesley. All rights reserved. Performance Issues Two performance criteria are placed on the system: Update rate for generating the augmenting image Accuracy of the registration of the real and virtual image –Update rate can limit registration accuracy as well –Brooks paper – “1 ms = 1mm error”

1-10 © 2010 Pearson Addison-Wesley. All rights reserved. Failures in Registration Failures in registration due to: –Noise Position and pose of camera with respect to the real scene –Image distortions –Time delays In calculating the camera position

1-11 © 2010 Pearson Addison-Wesley. All rights reserved. Display Technologies Monitor Based –Laptops –Cell phones –Projectors (more Ubiquitous Computing) Head Mounted Displays: –Video see-through –Optical see-through

1-12 © 2010 Pearson Addison-Wesley. All rights reserved. Monitor Based Augmented Reality Simplest available Treat laptop/PDA/cell phone as a window through which you can see AR world. Sunglasses demo

1-13 © 2010 Pearson Addison-Wesley. All rights reserved. Monitor Based AR Successful commercialization –Yellow line in football broadcasts –Glowing hockey puck –Replace times square billboards with own commercials during New Year’s Eve broadcasts –Baseball cards –Ad campaigns

1-14 © 2010 Pearson Addison-Wesley. All rights reserved. Optical see-through HMD

1-15 © 2010 Pearson Addison-Wesley. All rights reserved. Video see-through HMD

1-16 © 2010 Pearson Addison-Wesley. All rights reserved. Advantages of Video see-through HMD Flexibility in composition strategies Real and virtual view delays can be matched

1-17 © 2010 Pearson Addison-Wesley. All rights reserved. Advantages of Optical see-through HMD Simplicity Resolution No eye offset

1-18 © 2010 Pearson Addison-Wesley. All rights reserved. Advantage of Monitor Displays Consumer-level equipment Most practical A lot of current research aimed here Other current active area is a flip-down optical display.

1-19 © 2010 Pearson Addison-Wesley. All rights reserved. Early Application KARMA (91) –Feiner Optical see-through HMD Knowledge-based assistant for maintenance Ultrasound trackers attached to assembly parts

1-20 © 2010 Pearson Addison-Wesley. All rights reserved. Early Application Later – “architectural anatomy” Tourguide

1-21 © 2010 Pearson Addison-Wesley. All rights reserved. More Mechanical ECRC

1-22 © 2010 Pearson Addison-Wesley. All rights reserved. UNC - Medical Early 90’s Lots of work on reducing registration error Medical applications

1-23 © 2010 Pearson Addison-Wesley. All rights reserved. MIT Medical Laser-scanned patient LCD screen above patient

1-24 © 2010 Pearson Addison-Wesley. All rights reserved. AR Instructional Reality provides a natural interface –MagicBook

1-25 © 2010 Pearson Addison-Wesley. All rights reserved. AR Games- Toyota 86 (Android)

1-26 © 2010 Pearson Addison-Wesley. All rights reserved. AR Lots of new applications Will discuss some tracking systems next class