1 User Interfaces at Microsoft Research Intelligent Information Access using Animated 2 and 3D Information Visualization Mary Czerwinski

2 A Sweet Spot?  Use 3D Visuals  Apply 3D cues  Occlusion, Relative Size, Shadows  Spatialized Audio  2D Interaction technique  Little or no egocentric navigation  Objects come to the user  Use 3D Visuals  Apply 3D cues  Occlusion, Relative Size, Shadows  Spatialized Audio  2D Interaction technique  Little or no egocentric navigation  Objects come to the user

3 Data Mountain Subject Layout of 100 Pages “Strongest cue... relative size”

4 Video Data Mountain

5 Data Mountain Usability Study #1 (Compare with IE4 Favorites)  Reliably faster (26%) Study #2 (Longevity and Thumbnails)  After 6 months, no performance change  Images help, but are not required Study #1 (Compare with IE4 Favorites)  Reliably faster (26%) Study #2 (Longevity and Thumbnails)  After 6 months, no performance change  Images help, but are not required

6 “Thumbless” Data Mountain

7 Study #2 Results

8

9 Visualizing Implicit Queries  2 studies:  on management  on retrieval  Visualized with DM  3D, spatial layout of web pages  Pages similar to selected page are highlighted  2 studies:  on management  on retrieval  Visualized with DM  3D, spatial layout of web pages  Pages similar to selected page are highlighted

10 Implicit Queries (IQ): Performance Benefits  IQ users (IQ1 & 2)  built more categories  took longer organizing  faster finding pages  Retrieval far more frequent implies IQ faster overall IQ1,2 use different similarity metrics

11 Glances Novel 3D navigation technique  Lightweight, ephemeral Two-handed  Non-dominant hand used for glance gesture Low-cost awareness of surrounding area Novel 3D navigation technique  Lightweight, ephemeral Two-handed  Non-dominant hand used for glance gesture Low-cost awareness of surrounding area

12 Toolspaces  Objects attached to virtual body  Glances used to get to toolspaces  Multiple uses  3D widgets  Information store  Object transport  Objects attached to virtual body  Glances used to get to toolspaces  Multiple uses  3D widgets  Information store  Object transport

13 Task Gallery 3D Shell for Windows Task Management  Simple, forward-back navigation  Tasks laid out spatially on floor, ceiling, walls  Simple task switch 3D Shell for Windows Task Management  Simple, forward-back navigation  Tasks laid out spatially on floor, ceiling, walls  Simple task switch

14 Task Gallery Simultaneous viewing of multiple windows  Simple shift select  Smart arrangement  Use 3D to provide uniform scaling Simultaneous viewing of multiple windows  Simple shift select  Smart arrangement  Use 3D to provide uniform scaling

15 Task Gallery Tools  Glance Left for Start Palette  Glance Right for Windows Explorer  Glance Up or Down for status and notification  Glance Left for Start Palette  Glance Right for Windows Explorer  Glance Up or Down for status and notification

16 Task Gallery  Windows stacked on podium or in space until selected for use via “flicking”  User studies improved performance  Exploring other spaces/metaphors  Windows stacked on podium or in space until selected for use via “flicking”  User studies improved performance  Exploring other spaces/metaphors

17 Video Task Gallery

18 Hierarchy Visualization 2D and 3D Research Most frequently requested visualization solution from MS product teams Hierarchy Visualization 2D and 3D Research Most frequently requested visualization solution from MS product teams

19 Current Approaches Windows Tree Control  Many observed problems (from customers and usability testing) Many 2D and 3D hierarchy visualizations  Each works for some tasks and some scales Windows Tree Control  Many observed problems (from customers and usability testing) Many 2D and 3D hierarchy visualizations  Each works for some tasks and some scales

20 Problems: Fitting Data Extreme aspect ratio (broad and shallow) Balanced tree visualization not informative May be multiple hierarchies Scaling issues Extreme aspect ratio (broad and shallow) Balanced tree visualization not informative May be multiple hierarchies Scaling issues

21 Problems: Cognitive Overhead Loss of context when looking at detail Loss of detail when looking at overview Separate detail/overview – extra overhead Which item is open? Poor use of display space Loss of context when looking at detail Loss of detail when looking at overview Separate detail/overview – extra overhead Which item is open? Poor use of display space

22 Problems: Fitting User Task Multiple foci of interest is difficult No integration of tree view with search Tree structure may not reflect or support user task at hand Multiple foci of interest is difficult No integration of tree view with search Tree structure may not reflect or support user task at hand

23 Hierarchical Tree Control Problem

24 Basic View Strategies Two view (separate detail/overview views) Distorted view  Distorted data: fisheye  Distorted space: 3D, hyperbolic Focus in Context (integrated view) Two view (separate detail/overview views) Distorted view  Distorted data: fisheye  Distorted space: 3D, hyperbolic Focus in Context (integrated view)

25 Basic Visualization Approaches Indentation  Tree control  Fisheye Containment  Treemaps  Pad++ Indentation  Tree control  Fisheye Containment  Treemaps  Pad++ Clustering Galaxy of News Galaxy of News ThemeScapeThemeScape Hot SauceHot SauceGeographic Floor plans Floor plans Street mapsStreet maps

26 Basic Visualization Approaches Node-link diagrams  SemNet  2D diagrams  Cone Tree  Fisheye Cone Tree  Hyperbolic viewer  FSN  XML3D Node-link diagrams  SemNet  2D diagrams  Cone Tree  Fisheye Cone Tree  Hyperbolic viewer  FSN  XML3D

27 Perceptualization Goals Effective Use of Visual Perception Utilize Visual Perception Characteristics  Encode Documents along Multi-Dimensions  Add Shading/Transparency Cues to Increase Pre- attentive 3D Perception Audio Perception Haptic Perception Effective Use of Visual Perception Utilize Visual Perception Characteristics  Encode Documents along Multi-Dimensions  Add Shading/Transparency Cues to Increase Pre- attentive 3D Perception Audio Perception Haptic Perception

28 Perceptive Senses Available Ready to Use  Visual  3D Spatialization  Volume / Size  Color  Shape (curvature)  Opacity  Texture  Haptic Ready to Use  Visual  3D Spatialization  Volume / Size  Color  Shape (curvature)  Opacity  Texture  Haptic Experimental –Proprioception –Haptic –Auditory –Olfactory

29 XML3D

30 User Study-XML3D Used Snap.com content; compared XML3D to and 2D collapsible tree control UIUsed Snap.com content; compared XML3D to and 2D collapsible tree control UIwww.snap.com Target end users were developers and maintainers of large web info spacesTarget end users were developers and maintainers of large web info spaces Used search tasksUsed search tasks Single or multiple parentsSingle or multiple parents New of existing categoriesNew of existing categories Used Snap.com content; compared XML3D to and 2D collapsible tree control UIUsed Snap.com content; compared XML3D to and 2D collapsible tree control UIwww.snap.com Target end users were developers and maintainers of large web info spacesTarget end users were developers and maintainers of large web info spaces Used search tasksUsed search tasks Single or multiple parentsSingle or multiple parents New of existing categoriesNew of existing categories

31

32 Hierarchical Tree Control UI

33 Task Time Results: XML3D v. 2D UI’s

34 Consistency, Tool Usage Results For existing categories, XML3D benefits do not sacrifice quality of search Users preferred 2D lists for new category tasks. Why? For existing categories, XML3D benefits do not sacrifice quality of search Users preferred 2D lists for new category tasks. Why?

35 Direction… Using perceptual cues to provide important visual pattern information to user User can search or browse 3D final arrangement can emphasize multiple/user defined attributes Ready to drop in audio, haptic, clustering, implicit query Using perceptual cues to provide important visual pattern information to user User can search or browse 3D final arrangement can emphasize multiple/user defined attributes Ready to drop in audio, haptic, clustering, implicit query

36 3D UI Conclusions  Exploration of Desktop 3D “sweet spots”  Perform well against existing GUIs  Usability test informs design evolution  Audio in 3D environment is useful  Let users choose organization  Spatial memory and perceptual cues critical in Desktop 3D  Exploration of Desktop 3D “sweet spots”  Perform well against existing GUIs  Usability test informs design evolution  Audio in 3D environment is useful  Let users choose organization  Spatial memory and perceptual cues critical in Desktop 3D

37 3D User Interfaces Future Work  High value non-game apps:  Shell/Web/Mail/File Browser  New 3D interaction techniques  Navigation and manipulation  Input devices/techniques  New visualization techniques  Prove value of 3D and interactive animation  High value non-game apps:  Shell/Web/Mail/File Browser  New 3D interaction techniques  Navigation and manipulation  Input devices/techniques  New visualization techniques  Prove value of 3D and interactive animation