Presentation on theme: "Software Development Forum Emerging Technology SIG Weds. Sept 9 2009 Augmented Reality Clark Dodsworth Osage Associates dodsworth.com Maribeth."— Presentation transcript:
Software Development Forum Emerging Technology SIG Weds. Sept Augmented Reality Clark Dodsworth Osage Associates dodsworth.com Maribeth Back FX Palo Alto Laboratory fxpal.com
Theoretical foundations Scope My background Brief AR history Directions for development, revenue + evolution Maribeth Back, FX-PAL: “High Tech Chocolate: Exploring Mixed Reality applications for Industry” Ribbit.com demo by Kristi Wells and Brendan Lee Tonight’s Plan
Augmenting Reality and Revenue Connecting Mobile, Sensors, Location & Layers Clark Dodsworth dodsworth.com
Alfred North Whitehead, first augmented reality theorist: #1: “Civilization advances by extending the number of important operations which we can perform without thinking of them.”...effective AR doesn’t increase the cognitive load #2: We think in generalities, but we live in detail.”...AR tasks: managing detail #3: “The art of progress is to preserve order amid change and to preserve change amid order.”...contextually dynamic data management & display via AR
Scope - General AR = subset of Ubiquitous Computing: same infrastructure and issues as UbiComp and its personalization. AR = enhance personal awareness (as needed) of: physical & non- physical context, personal state, opportunities & threats...local & distant. AR = deliver personalized info with less effort, more space & time precision, and fewer seams than before, without increasing distraction.
Scope - General AR = subset of Ubiquitous Computing: same infrastructure and issues as UbiComp and its personalization. AR = enhance personal awareness (as needed) of: physical & non- physical context, personal state, opportunities & threats...local & distant. AR = deliver personalized info with less effort, more space & time precision, and fewer seams than before, without increasing distraction. Augmented Context + Information Salience = Value
Scope - Model 1st plateau of AR = augmented perception of internal, external, & distant reality. Smartphone and headset AR that’s non-intrusive. Beyond that, the value of well-evolved AR has significant parallels to the value of consciousness: 2nd plateau of AR = augmented decision-making. Integrated analysis, evaluation, and management (informatics) of data streams, constantly and dynamically evaluating the salience of each stream in context of the user’s ever-changing current situation and the user’s historical paths through n dimensions of data. Then displaying the most salient data...when appropriate: Contextual evaluation driving contextual augmentation; software ‘awareness’ augmenting the decision process via delivery of filtered salient aspects of the user’s situation. At least 2 generations further.
Scope - Model The value of well-evolved AR has significant parallels to the value of consciousness, AKA:...the Default Mode Network of the brain One of the hardest-working systems in the brain Active when the brain is aware, but not focused on a particular task Reduces activity when you concentrate on a specific task 2 linked brain areas: A)Imagination, empathy for the thoughts and intent of others B)Personal memories, visualization of future actions, scenarios
Scope - Model The value of well-evolved AR has significant parallels to the value of consciousness, AKA:...the Default Mode Network of the brain One of the hardest-working systems in the brain Active when the brain is aware, but not focused on a particular task Reduces activity when you concentrate on a specific task 2 linked brain areas: A)Imagination, empathy for the thoughts and intent of others B)Personal memories, visualization of future actions, scenarios DFM = constant process of considering how to be & what to do next. Getting and dealing with the data upon which to make all those decisions is the constant background task of 2nd-plateau AR apps.
Scope - Phone AR hot again as part of the smartphone ecosystem. Smartphone AR (SAR?) = low-hanging fruit. 2nd is games. SAR & LBS* = features, not an industry. Startups will fail / be acquired. Privacy / security issues will be more extreme than currently. Acronym Fusion: AR + LBS + GIS + Gesture = POTS... “my computer” *LBS = Location-Based Services
Scope - farther AR hot again as part of the smartphone ecosystem. Smartphone AR (SAR?) is the low-hanging fruit. SAR & LBS = features, not an industry; startups will fail / be acquired. Privacy / security issues will be more extreme. Acronym Fusion: AR + LBS + GIS = POTS... “my computer” After that: Contextual Personalization + More Sensors + onboard DBs + Projectors
Background My work Product strategy, dev., design, improvement, & IP evaluation, all based on User-Centered Design. For emerging technologies and markets in software, consumer electronics, and entertainment. My perspective “Product as Tool” Tool should adapt to (fit) the user and be its own user manual. Scythes are an excellent model system for tool design, though we don’t have ~2000 years to iterate.
AR-related projects: VideoCart, ‘92: Location-aware mobile retail touch-screen kiosk system with data-driven path mapping. Live Wall, ‘98: 100-meter, rear-projection, 1:1 scale videophone. “Ambient Intelligence” strategy,’98 for Philips Consumer Electronics. Royal Tyrell Museum, Alberta, ‘06: Concept & spec: permanent LBS- based virtual paleontological exhibit complex across 11 remote archaeological sites, for Reich + Petch. Dubailand theme parks, ‘08 : Concept evaluation & high-level system spec, strategic & tech partners: 24/7 multi-application smartphone AR experience. Background
VideOcart VideoCart: mobile location-aware touchscreen retail, ‘92
Live Wall: semi-mobile videophone design for Philips, ‘98
“Ambient Intelligence” strategy for Philips, ‘98 all photos: Philips
services across all hotel, leisure, retail, restaurant, and transit in Dubailand, and to follow the visitors home. Dubailand AR project, ‘08 Concept > smartphone AR Technology identification, assessment, & qualification High-level system definition Strategic partnering Broadened scope to include c. Lynne LaCascia, all rights reserved Part of Dubailand model:
AR Timeline Sailboat tell-tales ~1900 Rear-view mirrors 1st jet fighter heads-up display s Commercial aviation use Helmet- mounted displays “Augmented environments,” sensor nets US DoD air-combat testing st car HUD Olds Cutlass Supreme Now 20?? Battery energy- density resolved Horse carriages Tom Caudell & David Mizell ARToolkit UW HITLab Prehistory, soon after invention of the sail. Dedicated tools begin: “UbiComp”: Mark Weiser 2006 Useful smart- phone AR Nokia’s MARA Hands-up displays
Reed sails on boats in the prehistoric Middle East had inherent sail-edge tell-tales. Example of sensor/display fusion photo c. Luxury Peru Tours & Travel
Marmon Wasp racecar, 1911 Non-see-through HUD display: Rear-view mirror is just above line of sight
MIG21 heads-up display (installed): circa 1960s Cintel “Buccaneer” aircraft cockpit heads-up display device, UK 1958 photo: Rochester Avionics Archives, UK photo: Will Hise cc HUDs
HUD: modern commercial aviation AR
HMDs Helmet-mounted displays High-priority military R&D investment Tethered to power and computer Obstructs part of the field of view Delicate to align, adjust, maintain Size and weight unacceptable Adds complexity/distraction to user situation Current state of the art = net loss of context awareness c. Michael Edwards,2007 Rockwell-Collins Sim Eye SR100A, 2009
HMDs Helmet-mounted displays High-priority military R&D investment Tethered to power and computer Obstructs part of the field of view Delicate to align, adjust, maintain Size and weight unacceptable Adds complexity/distraction to user situation Current state of the art = net loss of context awareness Not AR c. Michael Edwards,2007 Rockwell-Collins Sim Eye SR100A, 2009
First car with production heads-up display: Oldsmobile 1988 Cutlass Supreme International Projected speedometer on inside of windshield. HUD
photo: Mobilizy GmbH Mobilizy's original product, “Wikitude AR (Travel) Guide,” for Android Hands-Up Displays
Some Players Mobilizy Wikitude World Browser, new Wikitude Drive (Android) Nokia Point & Find, Image Space, MARA Layar Layar Reality Browser 2.1 (Android) Enables adding layers easily Example: Trulia.com Working on iPhone dev now Studierstube ES Rich development framework Client and middleware 3D object tracking Copyright Network World, Inc. All rights reserved. photo: Mobilizy GmbH Total Immersion D’Fusion Studio, Pro, Mobile Metaio Unify Design: no programming Tonchidot Sekai Camera (iPhone & Android) Augmento....
Studierstube ES Copyright Network World, Inc. All rights reserved. photo: Mobilizy GmbH
Now GPS Accelerometers Light sensors Proximity sensors Compass Touch (2- or 2.5-D) Bluetooth Need Near-field RFID (“touch,” “wand”) Constant visual awareness + recording Constant audio awareness + recording Galileo-grade location resolution Ultrasonic & IR spatial analysis Laser projector ( s ) with geometric awareness & beam stabilization for object augmentation... Hardware
Next--- 1.Constant, robust markerless 2D & 3D feature I.D. and object recognition (Nokia Point & Find,...) as a general feature 2.Motion analysis + evaluation linked to the object recognition 3.General-purpose indoor position sensing 4.More nuanced gestural input interpretation + integration with voice 5.Constant audio + visual awareness with flagging & interpretation 1.Allostatic control of data delivery: apps “learn the user” over time, to drive what to display (‘awareness driving awareness,’ ‘context drives the augmentation of context’) 1.Constant, opportunistic acquisition and evaluation of available sensor-net feeds. Software
Still further out... Rob Cook’s award talk at Siggraph ‘09: “...the next holy grail for the computer graphics community should be realist augmented reality.” Software
Augmenting Revenue Assumptions Full OS citizenship: no tethering to a computer More coverage & bandwidth CPU & memory follow Moore’s Law Something magic happens to battery energy density. Work Extend existing software onto the mobile platform with AR where appropriate. Includes Augmenting Security for public locations. Building AR into new apps as a normal part of software develop- ment: Like the shift to web from desktop & enterprise. Multi-modal interface differentiation: Handheld vs. plugged into a desktop screen or large projector
Platforms and toolkits Platforms iPhone 3.9 Android 1.5 Palm WebOS RIM BlackBerry OS Symbian S60 5th Edition Windows Mobile 6.5 (Samsung LinuxOS?) Tools Studierstube ES Layar ARToolkit NYARToolkit FLARToolkit Tinmith...
dodsworth.com end OA logo
AR-related conferences Ubicomp 2009, Orlando, FL. Sept. 30-Oct. 3, Int’l Symposium on Mixed and Augmented Reality (ISMAR09) - Orlando, FL, October 19-22, th Int’l Conference on Mobile and Ubiquitous Multimedia (MUM 2009) Nov., Cambridge, UK 8th IEEE Int’l Conf. on Pervasive Computing & Communications (PerCom) Mannheim (Mar 29-April 2, 2010) 7th IEEE Workshop on Ubiquitous Communications & Services: Symposium on Wearable Computers (ISWC'09) - Linz, Austria, September 4-7, IEEE VR March 20-26, 2010 in Waltham, Massachusetts IEEE Symposium on 3D User Interfaces (3DUI), March , Waltham, Mass User Interfaces Symposium on Haptic InterfacesSymposium on Haptic Interfaces March , Waltham, Mass., Int’l Conference on Mobile, Wireless and Optical Communication (MWOC 2009)Int’l Conference on Mobile, Wireless and Optical Communication (MWOC 2009), Dec. 09 Korea Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) Nov. 30-Dec. 3 Orlando, FL SVC Wireless Annual Conference Sep 25-26, 2009, Mountain View Location Intelligence Conference 2009 Oct , Broomfield, CO 3rd Int’l Conference on Mobile Ubiquitous Computing, Systems, Services & Technologies UBICOMM October 11-16, in Sliema, Malta Mobiquitous nd ACM Int’l Workshop on Mobile Opportunistic Networking Feb , 2010, Pisa, 2nd Int’l Conference on Mobile Lightweight Wireless Systems (MOBILIGHT 2010) 5th Int’l Conference on Body Area Networks September (BodyNets) Sept , 2010 Corfu, Greece IMMERSECOM (2009 was at Berkeley)http://immerscom.org 1st Int’l ICST Conference on User Centric Media (UCMedia 2009), 9-11 Dec. 09, Venice