Presentation is loading. Please wait.

Presentation is loading. Please wait.

Software Development Forum Emerging Technology SIG Weds. Sept 9 2009 Augmented Reality Clark Dodsworth Osage Associates dodsworth.com Maribeth.

Similar presentations


Presentation on theme: "Software Development Forum Emerging Technology SIG Weds. Sept 9 2009 Augmented Reality Clark Dodsworth Osage Associates dodsworth.com Maribeth."— Presentation transcript:

1 Software Development Forum Emerging Technology SIG Weds. Sept Augmented Reality Clark Dodsworth Osage Associates dodsworth.com Maribeth Back FX Palo Alto Laboratory fxpal.com

2 Theoretical foundations Scope My background Brief AR history Directions for development, revenue + evolution Maribeth Back, FX-PAL: “High Tech Chocolate: Exploring Mixed Reality applications for Industry” Ribbit.com demo by Kristi Wells and Brendan Lee Tonight’s Plan

3 Augmenting Reality and Revenue Connecting Mobile, Sensors, Location & Layers Clark Dodsworth dodsworth.com

4 Alfred North Whitehead, first augmented reality theorist: #1: “Civilization advances by extending the number of important operations which we can perform without thinking of them.”...effective AR doesn’t increase the cognitive load #2: We think in generalities, but we live in detail.”...AR tasks: managing detail #3: “The art of progress is to preserve order amid change and to preserve change amid order.”...contextually dynamic data management & display via AR

5 Scope - General AR = subset of Ubiquitous Computing: same infrastructure and issues as UbiComp and its personalization. AR = enhance personal awareness (as needed) of: physical & non- physical context, personal state, opportunities & threats...local & distant. AR = deliver personalized info with less effort, more space & time precision, and fewer seams than before, without increasing distraction.

6 Scope - General AR = subset of Ubiquitous Computing: same infrastructure and issues as UbiComp and its personalization. AR = enhance personal awareness (as needed) of: physical & non- physical context, personal state, opportunities & threats...local & distant. AR = deliver personalized info with less effort, more space & time precision, and fewer seams than before, without increasing distraction. Augmented Context + Information Salience = Value

7 Scope - Model 1st plateau of AR = augmented perception of internal, external, & distant reality. Smartphone and headset AR that’s non-intrusive. Beyond that, the value of well-evolved AR has significant parallels to the value of consciousness: 2nd plateau of AR = augmented decision-making. Integrated analysis, evaluation, and management (informatics) of data streams, constantly and dynamically evaluating the salience of each stream in context of the user’s ever-changing current situation and the user’s historical paths through n dimensions of data. Then displaying the most salient data...when appropriate: Contextual evaluation driving contextual augmentation; software ‘awareness’ augmenting the decision process via delivery of filtered salient aspects of the user’s situation. At least 2 generations further.

8 Scope - Model The value of well-evolved AR has significant parallels to the value of consciousness, AKA:...the Default Mode Network of the brain One of the hardest-working systems in the brain Active when the brain is aware, but not focused on a particular task Reduces activity when you concentrate on a specific task 2 linked brain areas: A)Imagination, empathy for the thoughts and intent of others B)Personal memories, visualization of future actions, scenarios

9

10 Scope - Model The value of well-evolved AR has significant parallels to the value of consciousness, AKA:...the Default Mode Network of the brain One of the hardest-working systems in the brain Active when the brain is aware, but not focused on a particular task Reduces activity when you concentrate on a specific task 2 linked brain areas: A)Imagination, empathy for the thoughts and intent of others B)Personal memories, visualization of future actions, scenarios DFM = constant process of considering how to be & what to do next. Getting and dealing with the data upon which to make all those decisions is the constant background task of 2nd-plateau AR apps.

11 Scope - Phone AR hot again as part of the smartphone ecosystem. Smartphone AR (SAR?) = low-hanging fruit. 2nd is games. SAR & LBS* = features, not an industry. Startups will fail / be acquired. Privacy / security issues will be more extreme than currently. Acronym Fusion: AR + LBS + GIS + Gesture = POTS... “my computer” *LBS = Location-Based Services

12 Scope - farther AR hot again as part of the smartphone ecosystem. Smartphone AR (SAR?) is the low-hanging fruit. SAR & LBS = features, not an industry; startups will fail / be acquired. Privacy / security issues will be more extreme. Acronym Fusion: AR + LBS + GIS = POTS... “my computer” After that: Contextual Personalization + More Sensors + onboard DBs + Projectors

13 Background My work Product strategy, dev., design, improvement, & IP evaluation, all based on User-Centered Design. For emerging technologies and markets in software, consumer electronics, and entertainment. My perspective “Product as Tool” Tool should adapt to (fit) the user and be its own user manual. Scythes are an excellent model system for tool design, though we don’t have ~2000 years to iterate.

14 AR-related projects: VideoCart, ‘92: Location-aware mobile retail touch-screen kiosk system with data-driven path mapping. Live Wall, ‘98: 100-meter, rear-projection, 1:1 scale videophone. “Ambient Intelligence” strategy,’98 for Philips Consumer Electronics. Royal Tyrell Museum, Alberta, ‘06: Concept & spec: permanent LBS- based virtual paleontological exhibit complex across 11 remote archaeological sites, for Reich + Petch. Dubailand theme parks, ‘08 : Concept evaluation & high-level system spec, strategic & tech partners: 24/7 multi-application smartphone AR experience. Background

15 VideOcart VideoCart: mobile location-aware touchscreen retail, ‘92

16 Live Wall: semi-mobile videophone design for Philips, ‘98

17 “Ambient Intelligence” strategy for Philips, ‘98 all photos: Philips

18 services across all hotel, leisure, retail, restaurant, and transit in Dubailand, and to follow the visitors home. Dubailand AR project, ‘08 Concept > smartphone AR Technology identification, assessment, & qualification High-level system definition Strategic partnering Broadened scope to include c. Lynne LaCascia, all rights reserved Part of Dubailand model:

19 Dubailand AR project, ‘08 c. Lynne LaCascia, all rights reserved All images © Chimera Design

20 AR Timeline Sailboat tell-tales ~1900 Rear-view mirrors 1st jet fighter heads-up display s Commercial aviation use Helmet- mounted displays “Augmented environments,” sensor nets US DoD air-combat testing st car HUD Olds Cutlass Supreme Now 20?? Battery energy- density resolved Horse carriages Tom Caudell & David Mizell ARToolkit UW HITLab Prehistory, soon after invention of the sail. Dedicated tools begin: “UbiComp”: Mark Weiser 2006 Useful smart- phone AR Nokia’s MARA Hands-up displays

21 Reed sails on boats in the prehistoric Middle East had inherent sail-edge tell-tales.  Example of sensor/display fusion photo c. Luxury Peru Tours & Travel

22 Marmon Wasp racecar, 1911 Non-see-through HUD display: Rear-view mirror is just above line of sight

23 MIG21 heads-up display (installed): circa 1960s Cintel “Buccaneer” aircraft cockpit heads-up display device, UK 1958 photo: Rochester Avionics Archives, UK photo: Will Hise cc HUDs

24 HUD: modern commercial aviation AR

25 HMDs Helmet-mounted displays High-priority military R&D investment Tethered to power and computer Obstructs part of the field of view Delicate to align, adjust, maintain Size and weight unacceptable Adds complexity/distraction to user situation Current state of the art = net loss of context awareness c. Michael Edwards,2007 Rockwell-Collins Sim Eye SR100A, 2009

26 HMDs Helmet-mounted displays High-priority military R&D investment Tethered to power and computer Obstructs part of the field of view Delicate to align, adjust, maintain Size and weight unacceptable Adds complexity/distraction to user situation Current state of the art = net loss of context awareness  Not AR c. Michael Edwards,2007 Rockwell-Collins Sim Eye SR100A, 2009

27 First car with production heads-up display: Oldsmobile 1988 Cutlass Supreme International Projected speedometer on inside of windshield. HUD

28 photo: Mobilizy GmbH Mobilizy's original product, “Wikitude AR (Travel) Guide,” for Android Hands-Up Displays

29 Some Players Mobilizy Wikitude World Browser, new Wikitude Drive (Android) Nokia Point & Find, Image Space, MARA Layar Layar Reality Browser 2.1 (Android) Enables adding layers easily Example: Trulia.com Working on iPhone dev now Studierstube ES Rich development framework Client and middleware 3D object tracking Copyright Network World, Inc. All rights reserved. photo: Mobilizy GmbH Total Immersion D’Fusion Studio, Pro, Mobile Metaio Unify Design: no programming Tonchidot Sekai Camera (iPhone & Android) Augmento....

30 Studierstube ES Copyright Network World, Inc. All rights reserved. photo: Mobilizy GmbH

31 Now GPS Accelerometers Light sensors Proximity sensors Compass Touch (2- or 2.5-D) Bluetooth Need Near-field RFID (“touch,” “wand”) Constant visual awareness + recording Constant audio awareness + recording Galileo-grade location resolution Ultrasonic & IR spatial analysis Laser projector ( s ) with geometric awareness & beam stabilization for object augmentation... Hardware

32 Next--- 1.Constant, robust markerless 2D & 3D feature I.D. and object recognition (Nokia Point & Find,...) as a general feature 2.Motion analysis + evaluation linked to the object recognition 3.General-purpose indoor position sensing 4.More nuanced gestural input interpretation + integration with voice 5.Constant audio + visual awareness with flagging & interpretation 1.Allostatic control of data delivery: apps “learn the user” over time, to drive what to display (‘awareness driving awareness,’ ‘context drives the augmentation of context’) 1.Constant, opportunistic acquisition and evaluation of available sensor-net feeds. Software

33 Still further out... Rob Cook’s award talk at Siggraph ‘09: “...the next holy grail for the computer graphics community should be realist augmented reality.” Software

34 Augmenting Revenue Assumptions Full OS citizenship: no tethering to a computer More coverage & bandwidth CPU & memory follow Moore’s Law Something magic happens to battery energy density. Work Extend existing software onto the mobile platform with AR where appropriate. Includes Augmenting Security for public locations. Building AR into new apps as a normal part of software develop- ment: Like the shift to web from desktop & enterprise. Multi-modal interface differentiation: Handheld vs. plugged into a desktop screen or large projector

35 Platforms and toolkits Platforms iPhone 3.9 Android 1.5 Palm WebOS RIM BlackBerry OS Symbian S60 5th Edition Windows Mobile 6.5 (Samsung LinuxOS?) Tools Studierstube ES Layar ARToolkit NYARToolkit FLARToolkit Tinmith...

36 dodsworth.com end OA logo

37 AR-related conferences Ubicomp 2009, Orlando, FL. Sept. 30-Oct. 3, Int’l Symposium on Mixed and Augmented Reality (ISMAR09) - Orlando, FL, October 19-22, th Int’l Conference on Mobile and Ubiquitous Multimedia (MUM 2009) Nov., Cambridge, UK 8th IEEE Int’l Conf. on Pervasive Computing & Communications (PerCom) Mannheim (Mar 29-April 2, 2010) 7th IEEE Workshop on Ubiquitous Communications & Services: Symposium on Wearable Computers (ISWC'09) - Linz, Austria, September 4-7, IEEE VR March 20-26, 2010 in Waltham, Massachusetts IEEE Symposium on 3D User Interfaces (3DUI), March , Waltham, Mass User Interfaces Symposium on Haptic InterfacesSymposium on Haptic Interfaces March , Waltham, Mass., Int’l Conference on Mobile, Wireless and Optical Communication (MWOC 2009)Int’l Conference on Mobile, Wireless and Optical Communication (MWOC 2009), Dec. 09 Korea Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) Nov. 30-Dec. 3 Orlando, FL SVC Wireless Annual Conference Sep 25-26, 2009, Mountain View Location Intelligence Conference 2009 Oct , Broomfield, CO 3rd Int’l Conference on Mobile Ubiquitous Computing, Systems, Services & Technologies UBICOMM October 11-16, in Sliema, Malta Mobiquitous nd ACM Int’l Workshop on Mobile Opportunistic Networking Feb , 2010, Pisa, 2nd Int’l Conference on Mobile Lightweight Wireless Systems (MOBILIGHT 2010) 5th Int’l Conference on Body Area Networks September (BodyNets) Sept , 2010 Corfu, Greece IMMERSECOM (2009 was at Berkeley)http://immerscom.org 1st Int’l ICST Conference on User Centric Media (UCMedia 2009), 9-11 Dec. 09, Venice

38 Credits Default Mode Network paper: Gusnard, DA, Raichle, ME: “Searching for a baseline: the resting human brain from the perspective of functional imaging.” Nature Reviews Neuroscience, 2001 “iLamps: Geometrically Aware and Self-Configuring Projectors,” Rascar, et al Commercial aviation HUD image courtesy Todd Lapin, cc MARA image courtest Nokia Research Handheld AR.net (http://handheldar.net/stbes.php) for Studierstube ES diagramhttp://handheldar.net/stbes.php Fighter jet HUD device image cc Photo of sail’s tell-tales courtesy Bill Gracey, cc Modern helmet-mounted see-through display courtesy Rockwell-Collins: "SIM EYE SR100-A” Photo of Ray Harroun’s Marmon Wasp racecar with mirror, cc The359 Vuzix Wrap 920AV and Tac-Eye GMD images c. Vuzix Inc. Theme park illustrations © Chimera Design, Photo of Dubailand model © Lynne LaCascia, 2009

39 Software Development Forum Emerging Technology SIG Augmented Reality fxpal.com Maribeth Back, Fuji-Xerox Palo Alto Labs “High Tech Chocolate: Exploring Mixed Reality applications for Industry”


Download ppt "Software Development Forum Emerging Technology SIG Weds. Sept 9 2009 Augmented Reality Clark Dodsworth Osage Associates dodsworth.com Maribeth."

Similar presentations


Ads by Google