1. Francois Rioux Frank Rudzicz Mike Wozniewski HCI Project Presentation McGill University, April 6 2004

2. :: Overview  Proposed to improve visualization of 3D design.  Application is *immersive*.  Allows user to place him/herself inside the 3D world they are designing.  Uses a *two handed* gesture-based interface.  Allows for real-time rendering of display.  no need to switch between design-mode and render-mode.

3. :: Usage  The system could benefit many disciplines:  Computer Assisted Design (CAD).  3D game design.  Architecture, engineering, 3D layout.  Etc.

4. :: Usage  In our prototype, we focus on “Interior Design”.  User can place models of furniture, decorations, & appliances in a room.  User can modify these models: Rotate & Translate Apply Textures/Colors Etc.  User can save/restore various configurations.  User can invite his/her client to navigate around room to gauge a sense of the space before construction begins.

5. :: Design Decisions Unique Design Decisions: o Bimanual (Two Handed) Interaction o Toolglass interface widgets

6. :: Bimanuality  Must consider the properties of bimanual (two-handed) interaction as design constraint:  There exists structure to bimanual manipulations: asymmetry & division of labor (Guiard 1987).  Non-preferred vs. preferred hand.  Preferred hand is typically organized relative to a dynamic frame of reference provided by the non- preferred hand.  Benefits: Cognitive load lessened (Leganchuck 1998). Performance increase (Buxton & Meyers 1986). Provides additional kinesthetic feedback.

7. :: Toolglasses  Toolglass metaphor (Bier et al. 1993).  Semi-transparent menu.  Positioned over a target using the non-preferred hand.  Preferred hand clicks “thru” the menu to apply an operation to the target.

8. :: Toolglasses  Toolglasses were chosen and designed as pie menus with handles.  Provided the affordance that one could “grab” them.  Provided the affordance of crosshair-like targeting of objects behind the toolglass. Toolglass Proposed by Bier et al. Our Concept

9. :: Design Decisions  Other design decisions:  Toolglass rack – instead of one toolglass, have many and separate similar tasks among them.  3DS models – allow users to import the common.3ds (3D Studio Max) file format.  High-level modelling – users perform high-level tasks such as placing 3D models, applying textures/colors etc. (As opposed to constructing models and scenes from low-level primatives).

10. :: Initial Prototype  Coded in OpenGL.  Allowed modifications (coloring, scaling, rotation, translation) of GLUT primitives (sphere, cube, torus, teapot).

11. :: Prototype Evaluation Suggested changes:  Add visual feedback to show which model is currently targeted.  Add yes/no confirmation for irreversible actions (such as the reset command).  Add undo/redo.  Implement collision detection so models and user cannot pass through the walls of the world.  Instead of translating models with toolglasses, allow user to simply ‘grab’ the model and move it.  Differentiate ‘system’ toolglasses from ‘model’ toolglasses.

12. :: Alpha System  Many of the above changes were implemented.

13. :: Alpha System  Different toolglass handles:  Included support for importing.3DS (3D Studio Max) models, although texture support was buggy and disabled for the release: Modification toolglassesSystem toolglasses

14. :: Alpha System Also:  Added XML config files to configure toolglasses and options without need for recompiling. Adds flexibility to the design.  Added collision detection with walls. Added undo/redo:Added file saving:Added sub-toolglasses:

15. :: Beta System  Deployed in the SRE immersive environment, with gesture tracking.

16. :: Beta System  Added support for gesture tracking:  Toolglasses allowed keeping the gesture set at a minimum Pointing: Holding arm partially extended moves virtual cursors. Selection: Fully extending non-preferred hand selects a toolglass. Selection: Fully extending preferred hand invokes a wedge. Deselection: Arms at side of body drops toolglass.  Also, Navigation gesture: offsetting ones body from the center of the environment moves the camera in that direction.

17. :: Beta System Improved highlighting of targeted models: Ability to apply different textures and colors to sub-components of models:

18. :: Beta System Clarity & visibility improvement of toolglasses in rack: Confirmation of actions Before irreversible actions:

19. :: Future Work  Add support for placing pictograms/textures on toolglass wedges instead of text.  Improve tracking of gestures:  Employ a robust hand tracking algorithm rather than using ‘furthest point from center’ algorithm.  Use statistical tracking methods (eg. Condensation algorithm) to make tracking less susceptible to noise.  Experiments to test benefits of the interface and interaction techniques.

20. :: HCI Lessons  Visibility of system state & feedback was very important for users.  ie, knowing which model they were targeting, which toolglass wedge was selected, etc.  User feedback early in the process saved lots of time in the long term.  Eg: An initial user was concerned about being able to click on a wedge. We now use angular targeting, which was simpler to code and thus saved time.  A major challenge to gesture-based interfaces is tracking / recognizing the gestures.  Currently, we feel our design is robust yet poor gesture tracking makes it almost unusable.  It is important to prototype the physical aspects of a system.  It is quite physically demanding to keep one’s arms outstretched for a long time. Our prototype used a mouse and keyboard, so no users never tried performing our gesture set for an extended period of time.