Presentation on theme: "ARIS Augmented Reality Image Synthesis through illumination reconstruction and its integration in interactive and shared mobile AR-systems for E-(motion)-commerce."— Presentation transcript:
ARIS Augmented Reality Image Synthesis through illumination reconstruction and its integration in interactive and shared mobile AR-systems for E-(motion)-commerce applications 9 th E3 Concertation Meeting, Brussels, September 10 th, 2002 Mr. Ioannis Koufakis, INTRACOM S.A. Ikouf@intracom.gr
Introduction Goals of ARIS: –Provide new technologies for seamless integration of virtual objects in an augmented environment. –Develop new visualization and interaction paradigms for novel collaborative AR-applications. Two application scenarios will be developed: –Interactive desktop system. –Mobile AR-unit,that could be used on-site.
Collaboration –Between participants on-site –With a remote consultant (interior designer) in the store Realism enhancements –Physically correct placement of furniture (scaling, occlusion/collision detection etc.) –Modeling of lighting effects and correct visualization of virtual objects within the real environment (shading, illumination etc.)
Consortium House Market S.A.(IKEA)A7 Athens Technology Center P6 University of BristolP5 University of Manchester P4 Inria-LoriaP3 Intracom S.AP2 Fraunhofer InstituteC1 C - Coordinator P - Principal Contractor A - Assistant Contractor
Contract Information Contract No.: IST-2000-28707 Duration: 36 months Commencement: Oct. 2001 Current Stage: 12 th month Budget: ~4.9M Euro ( ~2,9M funded by EU) Web: aris-ist.intranet.gr
Geometry and Illumination Geometry Reconstruction –Scene and camera reconstruction, both from still images and videosequences. –Real-time camera tracking. Illumination Reconstruction –Reconstructing illumination and material properties from a sequence of images and 3D scene models. –Camera calibration:computation of camera response function.
Combined Lighting Simulation Algorithms for lighting simulation and image synthesis techniques for integration of virtual and real environments.
Perceptual Evaluation Development of a psychophysical framework measuring the perceptual realism in mixed/augmented reality scenes Tone mapping operators –Comparison of different tone mapping operators for both low and high dynamic range images. –Development of perceptual tone mapping operators.
Application Scenarios Specification of realistic application scenarios in an interior design application environment. User Requirements definition spanned two phases: –Initial phase, in order to guide research activities Questionnaires on ARIS web site were filled by visitors of the site Use case diagrams defined in UML notation –Final phase (due now), in order to capture the work situation in which the system will be deployed Special attention to HCI part A users’ workshop was organized internally in INTRACOM to evaluate the usability and acceptance of various tools.
Application Scenarios Survey of related systems and applications (deliverable D5.2) Results of the previous two were used to define the application scenarios and provide an initial specification of the system.
WP6: Application System Leaded by INTRACOM. Initial prototype, simulating AR on a computer screen will be operational in month 25. Final prototype at end of project will be a completed AR system. Current stage: architecture specification and system design (due to Oct.2002).
Application System 3D e-commerce server: repository of furniture models along with e-commerce functionality. Database was designed and implemented in Oracle 9i (ICOM,ATC). First approach of AR mobile unit was designed (ZGDV) –Marker based approach, for placement of furniture items within a scene –Active X component for visualization (AR browser), displaying VRML and live video. Wireless network, based on IEEE 802.11b.
Architecture Overview Interactive (web-based) application. Participants on-site 3D e-Commerce Server Authorization module Presentation Component Communication component Collaboration Gateway Repository DIM 3D models of furniture, textures etc. User profiles DIM Wearable Computer (coordinator) Tracking Visualization Communication Collaboration Wearable Computer (participant) Tracking Visualization Communication Broadband modem Access Point Broadband Network Local Server 3D Geometry Reconstruction module Illumination Reconstruction module 3D Geometry Reconstruction module Illumination Reconstruction module Collaboration Module
Demos Image synthesis using illumination reconstruction AR browser Interaction with Joystick Interaction with Voice Recognition
Problems so far User Interaction required for calibration and 3D reconstruction is too much (esp. for one of the techniques) Vision based techniques should be investigated to automate the process. Illumination reconstruction with the light probe is a strict procedure and requires knowledge of digital cameras and photography Automating detection of light sources should be pursued.
Problems so far Extension of those techniques to mobile AR-unit is not easy, especially due to limitations of mobile devices(small display, lack of keyboard, not enough computing power,etc.) Introduction of local server on-site, where an expert could operate the geometry and illumination modules. New interaction paradigms e.g voice,gesture recognition VRML not capable of handling illumination data and perform rendering of shadows.
Future Work Finalization of the system architecture and design. Investigate on how to minimize /improve user interaction. Collaboration paradigms: –Platforms to be used: NetMeeting, VoIP, Whiteboards, chat, Shared 3D Worlds etc. Investigate how wearable machines could be exploited. X-3D standardization effort.
Cross-project collaboration Exchange of experiences, especially for the mobile AR unit, with other projects –LifePlus –Archeoguide –Arvika (funded by German Ministry of Education and Research) Interaction paradigms within 3D worlds for mixed/augmented environments –Clustering