Chapter 2. 3D User Interfaces: History and Roadmap

Describe some of the history of 3D Uis Present a 3D UI “roadmap” that positions the topics covered in this book.

2.1 History of 3D UIs Prior to 1980 : CUI Around 1980 : GUI  Several technologies, such as the mouse, inexpensive raster graphics display, PC, and etc.  UI design and HCI became a much more important

2.1 History of 3D UIs 3D UI development  3D graphics technology, augmented reality (AR), and VE, flight simulator technology  New types of applications, leading to previously unexplored user tasks, new challenges in UI design, and unforeseen usability issues.

2.1 History of 3D UIs Ivan Sutherland (1968)  First head-tracked, head-mounted display First Virtual Environment  Jim Foley’s article in Scientific American(1987)  3D stereoscopic computer graphics, miniature CRT displays, position tracking systems, and interaction devices such as VPL DataGlove Intro.ppt

2.1 History of 3D UIs Advanced VE  Visualization of 3D scientific data sets  Real-time walkthroughs of architectural structures  VE games Research challenges  Faster, more realistic graphics, more accurate head- tracking, lower latency, better VE software toolkits  These application relatively impoverished when it came to user interaction.

2.2 Roadmap to 3D UIs 3 Areas  What types of works are parts of the 3D UI area  What disciplines and areas make up the background for 3D UI work  What impact 3D UIs have on other areas At least one important reference for most topics.

3D UIs Areas Influencing 3D UIs Areas Impacted by 3D UIs Th eoretic and Social background Human spatial perception, cognition, and action HCI and UI design Popular Media Technical background Interactive 3D Graphics 3D Visualiization 3D Input Device 3D Display Devices Simulator systems Telepresence systems Virtual and mixed reality systems 3D Interaction tech. and interface comp Interaction tech. for universal tasks Interaction tech. for complex or composite tasks 3D interaction techniques using 2D device 3D UI widgets 3D User Evaluation Evaluation of Devices Interaction techniques Complete 3D Uis or applications Specialized approaches 3D UI design approach Hybrid IT Two-handed Interaction Multimodal Interaction 3D Interaction aids 3D UI design strategies 3D UI Software tools Development tools for 3D applications Specialized development tools for 3D interaction 3D modeling tools Application Areas Simulation and training Education Entertainment, Art Visualization Architecture & construction Medicine and psychiatry Collaboration Standards For interactive 3D graphics For UI description Reciprocal impacts On graphics On HCI On psychology

2.2.1 Areas Informing the Design of 3D UIs Theoretical and Social Background  Human spatial perception, cognition, and action Psychology and human factors knowledge about the spatial perception; spatial cognition; and human navigation, movement, manipulation in 3D space contain critical background information for 3D UI design.(P15)  Basic principles of HCI and UI design A great deal of knowledge, theory has been generated by researchers in HCI : focused on desktop Uis. Applicable to 3D UIs (P15)  Popular media Different source of inspiration and vision for 3D UI Star Trek: The next generation, “Holodeck” The Matrix Snow Crash(Stephenson, 1992) : “cyberspace”

2.2.1 Areas Informing the Design of 3D UIs (continue) Technical Background  Interactive 3D graphics (P16 참조 ) Producing realistic but synthetic 3D images Interactive 3D graphics – rendered in real time  3D visualization (P17 참조 ) Changing abstract data into perceptual form  3D input devices (Chapter 4) Tracking device, Multiple-DOF joysticks, Isotonic 3D input device  3D display devices (Chapter 3) Stereo viewing Immersion 3D spatial sound system  Simulator systems Before VR, simulator systems pioneered the use of large, interactive displays Flight simulation, tank and military vehicle simulation, space vehicle simulation, and simulator for entertainment  Tele-presence systems Allow one user in one real world location to feel as if he were in a different real-world location. Sensors (cameras, microphone), display (visual, auditory, haptic), and interactive controls  Virtual Reality systems Sutherland’s HMD system VPS’s HMD and DataGlove The Cave Automatic Virtual Environment(CAVE)

D UI Sub areas (1) 3D Interaction Techniques & Interface Components  Just 2D UIs are built from components like windows, scrollbars, menus, and drag-and-drop, 3D UIs are built from large number of techniques and components Interaction techniques for universal tasks Interaction techniques for complex or composite tasks 3D interaction techniques using 2D devices 3D UI widgets

D UI Sub areas Interaction techniques for universal tasks (chapters 5-9) Selection, travel, wayfinding, system control, and system input are common and low-level user tasks. Interaction techniques for complex or composite tasks More complex tasks in 3D UIs are composed of the universal tasks described above.

D UI Sub areas 3D interaction techniques using 2D devices (chapter 5, section 5.4.7)  In a 3D modeling programs running on desktop computers, we need a way to map 2D mouse input to the 6 DOF of a 3D object. 3D UI widgets (Chapter8, section 8.3)  Virtual knife  Small icon

D UI Sub-areas (2) 3D UI Design Approaches  We need higher-level approach or strategies for building 3D interfaces : from low level interaction techniques and interfaces components Hybrid interaction techniques Two-handed interaction Multimodal interaction 3D interaction aids 3D UI design strategies

D UI Sub-areas Hybrid interaction techniques  HOMER manipulation techniques from ray-casting and arm extension Two-handed interaction (Chapter 10, section )  Allow the user to use both hands in a complementary way Multimodal interaction (Chapter 8, section 8.7)  Combining hand-based gestures with speech input 3D interaction aids  Physics: Forces, collisions, etc.  Constraints: snapping, grouping, etc. 3D UI design strategies (Chapter 10)

D UI Sub areas (3) 3D UI Software Tools  Tools are needed to turn conceptual UI design into concrete prototypes and implementations. Development tools for 3D applications Specialized development tools for 3D interfaces 3D modeling tools

D UI Sub areas Development tools for 3D applications  Software Libraries, Toolkits, application programming interfaces (APIs) and IDEs  PL such as C++, APIs for OpenGL, etc. Specialized development tools for 3D interfaces  VRML, Extensible 3D(X3D) 3D modeling tools  AutoCAD, 3D Studio Max, and Maya.

D UI Sub areas (4) 3D UI Evaluation  helps designers pursue good ideas and reject poor ones,  compare two or more alternatives for a particular UI components,  validate the usability of a complete application, and more. Evaluation of devices Evaluation of interaction techniques Evaluation of complete 3D UIs or applications Specialized evaluation approaches Studies of phenomena particular to 3D UIs

D UI Sub areas Evaluation of devices Evaluation of interaction techniques  When there are many possible techniques for a particular task, a comparative evaluation can reveal the tradeoffs in usability and performance between techniques. Evaluation of complete 3D UIs or applications Specialized evaluation approaches  Generic methodologies for evaluating the usability of 3D interfaces.  Testbed evaluation: Chapter 11, section Studies of phenomena particular to 3D Uis  Presence  Cybersickness  Precise definitions and methods for measuring it, and metrices are beginning to emerge. Immersion.ppt

2.2.3 Areas impacted by 3D UIs Application Areas  Simulation and training  Education  Entertainment  Art  Visualization  Architect and construction  Medicine and psychiatry  Collaboration Application.ppt

2.2.3 Areas impacted by 3D UIs Standards  For interactive 3D graphics  For UI description

2.2.3 Areas impacted by 3D UIs Reciprocal Impacts  On graphics  On HCI  On psychology

2.3 Scope of this book