1. State-of-the-art report in Augmented Reality authoring/modelling/presenting Eng. Pedro Santos pedro.santos@iscte.pt Miguel Dias Miguel.dias@iscte.pt ADETTI/ISCTE Jan 2005 This work is part of an Msc thesis in Computer and Telecom. Engineering - Multimedia... ART02 / ISMAR02 / ART03 / ISMAR03 / ISMAR04...

2. ART02

3. AR-based Product Design in Automobile Industry This paper describes different scenarios for the use of augmented reality in the early design phase for new cars. The prototype uses different tracking systems like AR-ToolKit, Polhemus Fastrack and Pinch Gloves for user interaction. Dipl.-Inform. Juergen Fruend Dipl.-Inform. Carsten Matysczok Cand. Ing. Rafael Radkowski Heinz Nixdorf Institute, University of Paderborn Fuerstenallee 11, Paderborn, 33102, Germany {fruend, onestone, rafael}@hni.uni-paderborn.de

4. ISMAR02

5. Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design Spacedesign is an innovative Mixed Reality (MR) application addressed to aesthetic design of free form curves and surfaces. It is a unique and comprehensive approach which uses task-specific configurations to support the design workflow from concept to mock-up evaluation and review. The first-phase conceptual design benefits from a workbench-like 3-D display for free hand sketching, surfacing and engineering visualization. Semitransparent stereo glasses augment the pre-production physical prototype by additional shapes, textures and annotations. Both workspaces share a common interface and allow collaboration and cooperation between different experts, who can configure the system for the specific task. A faster design workflow and CAD data consistency can be thus naturally achieved. Michele Fiorentino DIMeG, Politecnico di Bari, Italy fiorentino@dimeg.poliba.it Raffaele de Amicis GRIS,Universität Darmstadt, Germany ramicis@igd.fhg.de Giuseppe Monno d.Dis, Politecnico di Bari, Italy gmonno@poliba.it Andre Stork IGD A2, Darmstadt, Germany stork@igd.fhg.de

6. Augmented Urban Planning Workbench: Overlaying Drawings, Physical Models and Digital Simulation There is a problem in the spatial and temporal separation between the varying forms of representation used in urban design. Sketches, physical models, and more recently computational simulation, while each serving a useful purpose, tend to be incompatible forms of representation. The contemporary designer is required assimilate these divergent media into a single mental construct and in so doing is distracted from the central process of design. We propose an Augmented Reality Workbench called “Luminous Table” that attempts to address this issue by integrating multiple forms of physical and digital representations. 2D drawings, 3D physical models, and digital simulation are overlaid into a single information space in order to support the urban design process. Hiroshi Ishii, John Underkoffler Dan Chak, Ben Piper Tangible Media Group, MIT Media Laboratory {ishii, jh, chak, benpiper}@media.mit.edu Eran Ben-Joseph, Luke Yeung*, Zahra Kanji Department of Urban Studies and Planning Department of Architecture* MIT School of Architecture and Planning {ebj, lyeung, zahra}@MIT.EDU

7. Alternative Tools for Tangible Interaction: A Usability Evaluation In this work we compare an in-house designed Tangible User Interface (TUI) with three alternative single-user tools through an empirical investigation. These three alternative tools are a 3D physical, a 2D cardboard, and a mathematical tool. We expected the 3D physical to perform best, followed by the TUI, the 2D cardboard, and the mathematical tool. A pilot study was first carried out, the results of which were used to design a major experiment. Participants solved the same positioning problem, each using one of the four tools. The mathematical tool was not used in the experiment. In the experiment, trial time, number of user operations, learning effect in both preceding variables, and user satisfaction were measured. Morten Fjeld, Sissel Guttormsen Schär, Domenico Signorello, Helmut Krueger IHA, Swiss Federal Institute of Technology Clausiusstr. 25, CH-8092 Zurich, Switzerland morten@fjeld.chmorten@fjeld.ch, guttormsen@iha.bepr.ethz.ch,guttormsen@iha.bepr.ethz.ch domenico.signorello@wcom.chdomenico.signorello@wcom.ch, krueger@iha.bepr.ethz.chkrueger@iha.bepr.ethz.ch

8. A pragmatic approach to Augmented Reality Authoring In this paper we describe the Augmented Reality (AR) authoring system "PowerSpace" which allows fast and comfortable generation of AR worlds. The system presented uses the functionality of a 2D presentation program (Microsoft PowerPoint) as the basis for the composition of 3D content. An MS PowerPoint export is used to generate an XML-based extensible description of a presentation. This description is enriched by 3D content with the help of an editor, which is also part of the PowerSpace system. The content of this presentation is finally converted into 3D scenes and is used in an AR-Viewer. Matthias Haringer Fachhochschule Ulm University of Applied Sciences P.O. Box 3860, 89028 Ulm / Germany +49 731 50208 Matthias.Haringer@gmx.net Holger T. Regenbrecht DaimlerChrysler AG, Research and Technology Virtual and Augmented Environments P.O. Box 2360, 89013 Ulm / Germany +49 731 505 4307 Holger.Regenbrecht@DaimlerChrysler.com

9. MR Platform: A Basic Body on Which Mixed Reality Applications Are Built This paper describes a platform package, called “MR Platform,” which we have been implementing for research and development of augmented reality technology and applications. This package includes a parallax-less stereo video see-through HMD and a software development kit (SDK) for a Linux PC environment. The SDK is composed of a C++ class library for making runtime MR applications and related utilities such as a camera calibration tool. By using the SDK, the following functions are available; capturing video, handling a six degree-offreedom (DOF) sensor, image processing such as color detection, estimating head position and orientation, displaying the real world image, and calibrating sensor placement and camera parameters of two cameras mounted on the HMD. Shinji Uchiyama, Kazuki Takemoto, Kiyohide Satoh, Hiroyuki Yamamoto, and Hideyuki Tamura MR Systems Laboratory, Canon Inc. 2-2-1 Nakane, Meguro-ku, Tokyo, 152-0031 Japan {uchiyama.shinji,takemoto.kazuki,sato.kiyohide,yamamoto.hiroyuki125}@canon.co.jp, HideyTamura@acm.org

10. Inexpensive Non-Sensor Based Augmented Reality Modeling of Curves and Surfaces in Physical Space Previous works in modeling of curves and surfaces in augmented reality (AR) space has used expensive sensors such as magnetic sensors. In this work, we propose an augmented reality system where a user can model interesting surfaces with her hands, and without expensive sensing systems. The system uses computer vision based methods for the tracking of the user’s head and hand position. Using a glove and the tracking system, the user can draw smooth lines or surfaces with her hands in a physical space. Also the user can intuitively modify the lines or surface created by pushing or pulling at the control points of the lines or curve in a tangible manner. Adrian David Cheok, Neo Weng Chuen Edmund and Ang Wee Eng National University of Singapore, http://mixedreality.nus.edu.sg

11. ART03

12. 3D-Registered Interaction- Surfaces in Augmented Reality Space This paper describes the ARGUI system, which provides developers of ARToolkit applications with the possibility to create 2D Interaction-Surfaces registered in 3D. With ARGUI 2D interactions on 2D objects registered in 3D are possible, e.g. attached to a marker. The integration of a complete 2D GUI library is shown in detail, which simplifies the creation of the user interface even more. Christian Geiger#, Leif Oppermann#, Christian Reimann*, #Hochschule Harz, *Paderborn University C-LAB cgeiger@hs-harz.decgeiger@hs-harz.de, u13859@hs-harz.de, reimann@c-lab.deu13859@hs-harz.dereimann@c-lab.de

13. A Dice Game in Third-Person Augmented Reality We describe a prototype entertainment application of the Augmented-Reality Toolkit based on a fantasy dice game. Two players roll dice bearing glyphs that are interpreted by a computer, which provides graphical and auditory feedback. Our prototype uses entirely consumergrade equipment: A USB webcam, a projector, and a 2 GHz desktop with 5.1 surround speakers. Unlike many AR-Toolkit applications, our players are not encumbered by head-mounted displays. Face-to-face gameplay, integrated with the physicality of a traditional dice game, display results on a shared projection screen from a third-person point-of-view. Richard Colvin, Ted Hung, David Jimison, Benjamin Johnson, Eben Myers, Tina Blaine Entertainment Technology Center Carnegie Mellon University 700 Technology Drive Pittsburgh, PA 15219 USA {rcolvin, tsh, djimison, bkj, emyers, tblaine}@andrew.cmu.edu

14. AR-based Modular Construction System for Automobile Advance Development This paper describes the development of an AR-based modular construction system, which can be used within the phases of automobile advance development. The application completes real automobile prototypes by virtual components to show design variants or to support design reviews. Therefore the user can select virtual car components out of a virtual component menu and place them on a real car. The interaction with the AR-scene is done by hand gestures. The evaluation of the applications was done at Volkswagen AG, department of commercial vehicles, a German automobile manufacturer. J. Gausemeier, C. Matysczok and R. Radkowski Department of Computer Integrated Manufacturing Heinz Nixdorf Institute, University of Paderborn Paderborn, Germany {juergen.gausemeier|carsten.matysczok|rafael.radkowski}@hni.upb.de

15. Collaborative Augmented Sketching The aim of this paper is to demonstrate a software prototype using AR Toolkit for collaborative augmented reality sketching in architectural design. I introduce a non-intrusive interaction technique developed for this prototype. Additionally, sketching and distribution mechanisms will be discussed and illustrated. The prototype uses non-photo-realistic rendering and an adaptive tessellation mechanism in the geometry kernel to provide a visual cue for the conceptual stage of an architectural design. Hartmut Seichter Department of Architecture The University of Hong Kong seichter@hku.hk

16. MagicCup: A Tangible Interface for Virtual Objects Manipulation in Table-Top In this demonstration, we show a tangible interface for virtual object manipulation in table-top augmented reality based on ARToolKit. This demonstration is designed for city planning. Augmented reality technology enables users to consider of city plans more effectively and easily. One important issue of the augmented reality environment is how user can manipulate 3D structures that are displayed as virtual objects. It has to be intuitive and easy so that it may not disturb user's thought. We propose a new direct manipulation method based on tangible user interface. User holds a transparent cup upside down and can pick up, move or delete a virtual object by using it. Hirokazu Kato, Keihachiro Tachibana, Masaaki Tanabe, Takeaki Nakajima, Yumiko Fukuda Osaka University, Hiroshima City University, Knack Images Production Center, Hiroshima Institute of Technology kato@sys.es.osaka-u.ac.jpkato@sys.es.osaka-u.ac.jp, tatibana@sys.im.hiroshima-cu.ac.jptatibana@sys.im.hiroshima-cu.ac.jp, nakajima@art.hiroshima-cu.ac.jpnakajima@art.hiroshima-cu.ac.jp, nachm@lime.ocn.ne.jp yfukuda@cc.it-hiroshima.ac.jp

17. ISMAR03

18. Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures Mixed Reality is a very useful and powerful instrument for the visualization of processes, including the assembly process. A Mixed Reality based step-by-step furniture assembly application is introduced. On the one hand context related actions are given to the user to install elements. On the other hand an intuitive way for authors to create new MR based assembly instructions is provided. Our goal is to provide a powerful, flexible and easy-to-use Authoring Wizard for assembly experts, allowing them to author their new assembly instructor for hierarchical structures. This minimizes the costs for the creation of new Mixed Reality Assembly Instructors. Jurgen Zauner, Michael Haller, Alexander Brandl amire@fh-hagenberg.at Upper Austria University of Applied Sciences (MTD) 4232 Hagenberg, Austria Werner Hartmann wh@faw.uni-linz.ac.at Institute for Applied Knowledge Processing (FAW) 4040 Linz, Austria

19. Interactive Mediated Reality Mediated reality describes the concept of filtering our vision of reality, typically using a head- worn video mixing display. In this paper, we propose a generalized concept and new tools for interactively mediated reality. We present also our first prototype system for painting, grabbing and glueing together real and virtual elements. Raphael Grasset, Jean-Dominique Gascuel iMAGIS/GRAVIR INRIA Rhone-Alpes 655, Avenue de l’Europe 38330 Montbonnot, France Raphael.Grasset,Jean-Dominique.Gascuel@imag.fr Dieter Schmalstieg Interactive Media System Group Vienna University of Technology Favoritenstrasse 9-11/188/2 A-1040 Vienna, Austria Schmalstieg@ims.tuwien.ac.at

20. The Augmented Composer Project: The Music Table The Music Table enables the composition of musical patterns by arranging cards on a tabletop. An overhead camera allows the computer to track the movements and positions of the cards and to provide immediate feedback in the form of music and on-screen computer generated images. Musical structure is experienced as a tangible space enriched with physical and visual cues about the music produced. Rodney Berry, Mao Makino, Naoto Hikawa, Masumi Suzuki ATR Media Information Science Laboratories rodney@atr.co.jp

21. ISMAR04

22. Collaborative Mixed Reality Visualization of an Archaeological Excavation We present VITA (Visual Interaction Tool for Archaeology), an experimental collaborative mixed reality system for offsite visualization of an archaeological dig. Our system allows multiple users to visualize the dig site in a mixed reality environment in which tracked, see-through, head-worn displays are combined with a multi- user, multi-touch, projected table surface, a large screen display, and tracked hand-held displays. We focus on augmenting existing archaeological analysis methods with new ways to organize, visualize, and combine the standard 2D information available from an excavation (drawings, pictures, and notes) with textured, laser rangescanned 3D models of objects and the site itself. Users can combine speech, touch, and 3D hand gestures to interact multimodally with the environment. Hrvoje Benko Edward W. Ishak Steven Feiner Department of Computer Science, Columbia University New York, NY 10027 {benko, ishak, feiner}@cs.columbia.edu www.cs.columbia.edu/graphics

23. Immersive Authoring of Tangible Augmented Reality Applications In this paper we suggest a new approach for authoring tangible augmented reality applications, called ‘immersive authoring.’ The approach allows the user to carry out the authoring tasks within the AR application being built, so that the development and testing of the application can be done concurrently throughout the development process. We describe the functionalities and the interaction design for the proposed authoring system that are specifically targeted for intuitive specification of scenes and various object behaviors. Several cases of applications developed using the authoring system are presented. A small pilot user study was conducted to compare the proposed method to a non-immersive approach, and the results have shown that the users generally found it easier and faster to carry out authoring tasks in the immersive environment. Gun A. Lee α Claudia Nelles β Mark Billinghurst β Gerard Jounghyun Kim α α Virtual Reality Laboratory, Pohang University of Science and Technology β Human Interface Technology Laboratory New Zealand, University of Canterbury α {endovert, gkim}@postech.ac.kr β {claudia.nelles, mark.billinghurst}@hitlabnz.org

24. Level of Detail Interfaces We present the novel level of detail interface based on the marriage of level of detail geometry and an adaptable user interface. Level of detail interfaces allow applications to paramaterize their display of data and interface widgets with respect to distance from the camera, to best take advantage of diminished screen space in a 3D environment. Stephen DiVerdi, Tobias H¨ollerer, Richard Schreyer Department of Computer Science University of California, Santa Barbara, CA 93106 {sdiverdi,holl,richards}@cs.ucsb.edu

25. Other projects Other projects

26. AMIRE The main objective of AMIRE is to enable non-expert researchers:  to use Mixed Reality (MR) for their applications and  to create and modify these MR applications with the support of dedicated tools that foster an efficient authoring process for MR. This is the key for a more widespread use of MR, for a transfer of MR into different application domains, for exploiting synergies between different MR methodologies and for establishing authoring itself as a new application domain for MR. With authoring tools that employ MR, users are given means to efficiently communicate their ideas. An objective of AMIRE is also to set up and exploit a business scenario where a MR provider produces MR content for a customer. http://www.amire.net/

27. DART: The Designer’s Augmented Reality Toolkit This demonstration will highlight the Designer’s Augmented Reality Toolkit (DART), a system that allows users to easily create augmented reality (AR) experiences. Over the past year our research has been focused on the creation of this toolkit that can be used by technologists, designers, and students alike to rapidly prototype AR applications. Current approaches to AR development involve extensive programming and content creation as well as knowledge of technical topics involving cameras, trackers, and 3D geometry. The result is that it is very difficult even for technologists to create AR experiences. Our goal was to eliminate these obstacles that prevent such users from being able to experiment with AR. The DART system is based on the Macromedia Director multimedia-programming environment, the defacto standard for multimedia content creation. DART uses the familiar Director paradigms of a score, sprites and behaviors to allow a user to visually create complex AR applications. DART also provides low-level support for the management of trackers, sensors, and camera via a Director plug-in Xtra. http://www.gvu.gatech.edu/dart/ Blair MacIntyre, Maribeth Gandy, Jay Bolter, Steven Dow, Brendan Hannigan GVU Center, Georgia Institute of Technology, Atlanta, GA (blair, maribeth,steven,brendan)@cc.gatech.edu, jay.bolter@lcc.gatech.edu

28. DesignAR DesignAR allows a way of authoring augmented reality environments without needing to write code in C++ or other programming languages. The environment centers around Touch Designer, a commercial 3D modelling package by Derivative www.derivativeinc.comwww.derivativeinc.com Based on the MXR toolkit programming libraries, our program uses a video camera to track special markers and send their positions and rotations to Touch Designer. The models are positioned in relation to the 'virtual camera' in Touch Designer and placed in front of a background of live video. This creates the illusiion of CG objects in the real world. Touch designer allows complex interaction to be developed on-the-fly. The image below shows a particle system of envelopes being blown by a rotating fan. The fan and the particles can be moved around by manipulating the markers. http://www.mis.atr.co.jp/~rodney/designAR/index.htm

29. The MagicBook - Moving Seamlessly between Reality and Virtuality Mark Billinghurst University of Washington Hirukazu, Kato Hiroshima City University Ivan Poupyrev Sony Computer Science Laboratories

30. MagicMouse: an Inexpensive 6-Degree-of-Freedom Mouse An inexpensive computer input device was developed that allows the user to operate within both 2D and 3D environments by simply moving and rotating their fist. Position and rotation around the X, Y and Z-axes are supported, allowing full six degree of freedom input. This is achieved by having the user wear a glove, to which is attached a square marker. Translation and rotation of the hand is tracked by a camera attached to the computer, using the ARToolKit software library. Extraction, calibration, normalisation and mapping of the data converts hand motion into meaningful operations within 2D and 3D environments. Four input scenarios are described, showing that the mapping of the position and rotation data to 2D or 3D operations depends heavily on the desired task. Eric Woods HIT Lab NZ eric.woods@hitlabnz.org Paul Mason Lincoln University, New Zealand masonp3@lincoln.ac.nz Mark Billinghurst HIT Lab NZ mark.billinghurst@hitlabnz.org