1. From Hierarchies to Polyarchies: Visualizing Multiple Relationships
George G. Robertson
Microsoft Research

2. What is the problem? Hierarchies are very common
20 years of hierarchy visualization R&D
Significant problems remain
New problems appearing (Multiple Hierarchies)
Hierarchies (or trees) are one of the most common information structures
A typical computer user probably interacts with hierarchies dozens to hundreds of times each day
Distinguish between networks, DAG’s, and hierarchies
Network – general graph (e.g., the web)
DAG – Directed Acyclic Graph - a subset; no cycles
Hierarchy – a subset; each node has only one parent
Hierarchy visualization is also important because it can be used to view Networks and DAGs with some preprocessing
Over the last 20 years there has been a lot of research on how to effectively display and interact with hierarchies
In spite of that, we still have not solved some basic problems
In addition, as the industry scales to larger enterprise-wide problems, we are beginning to see uses of multiple hierarchies
Which current approaches do not handle
At Microsoft Research, we set out last Fall to understand current problems with Hierarchy Visualization and propose new solutions
I’m going to tell you about of what we have found so far

3. Current Approaches Many 2D and 3D hierarchy visualizations
Each works for some tasks and some scales
Very few have had user testing
Windows Tree Control
Many observed problems
Many hierarchy visualization techniques have been proposed over the last 20 years
Each solves some problems
None solve all problems
Note that very few of these have actually been tested with real users
I’ll show you a survey of some of these techniques
Windows Tree Control is most commonly used hierarchy visualization
Many problems are reported by customers and observed in formal usability testing
Most frequently requested visualization solution from MS product teams

4. What’s wrong with this picture?
The Windows Tree Control
It is used in many different ways: Windows Explorer, file open/close dialogue, History, Favorites, …
It does some things well –
It can be used in a very small space.
But, it has a lot of problems …

5. Problems: Fitting Data
Extreme aspect ratio (broad and shallow)
May be multiple hierarchies
Scaling issues
Two classes of problems: The first is how well the visualization fits the data
From our research, we see that most real-world hierarchies tend to be very broad and shallow
For example, looking at either file systems or mail folders, we see not many levels, but moderate to high fan-out
We are beginning to see problems that involve more than one hierarchy, but most tools only show one.
For example, an enterprise may have many databases about people, and want to view them together
We know that a given visualization works well within some limits on the number of nodes or levels,
But probably does not work well when the data scales beyond those limits

6. Problems: Cognitive Overhead
Loss of context
Or loss of detail
Separate detail/overview  extra overhead
Multiple focus is difficult
Which item is open?
The second class has to do with the cognitive demands placed on the user by the visualization
For any hierarchy more than about 80 nodes, it is very difficult to show all the details on one screen
Leads to either a loss of context or a loss of detail
A common early solution was to provide two views, one for details and one for an overview
But, this forces the user to shift focus of attention from one place to another
And can lead to losing context
Many user tasks require looking at more than one thing at a time, but most current interfaces do not allow multiple focus
Once a node has been selected (or opened), and the user scrolls away from it, the user quickly loses track of where the selected node is

7. Basic View Strategies Two view (separate detail/overview views)
Distorted view
Distorted data: fisheye
Distorted space: 3D, hyperbolic
Focus in Context (integrated view)
As we begin to look at the history of visualization techniques, you will see three basic view management strategies
Two view – separate detail and overview views
Distortion
Of the data (e.g., filtering out less interesting material)
Of the space
Focus in context displays provide a seamless integration of detail and context

8. Basic Visualization Approaches
Indentation
Tree control
Fisheye
Containment
Treemaps
Pad++
Clustering
Galaxy of News
ThemeScape
Hot Sauce
Geographic
Floor plans
Street maps
Node-link diagrams
2D diagrams
SemNet
Cone Tree
Fisheye Cone Tree
Hyperbolic viewer
FSN
XML3D
Five basic approaches to visualizing hierarchies
Indentation
The early tree views used this
Fisheye added a distortion technique to increase amount of useful information on screen
Containment
Treemaps are a 2D space-filling visualization of hierarchies using containment
Pad++ is a 2D plus zoom containment system (where some levels are hidden because of zoom factor)
Clustering data
These techniques are for implicit hierarchies derived from content analysis
Geographic
Sometimes hierarchy has a direct geographic mapping or analogy
Node-link diagrams
Probably the most explored in the Information Visualization community
Let’s go through some examples of these in chronological order

9. Smalltalk File Browser - 1979
Xerox PARC 1979
One of the earliest hierarchy visualizations was the Smalltalk File Browser in 1979
Four levels of the hierarchy are shown, but only siblings are shown at each level

10. Fisheye Views – 1986 Furnas, Bell Labs report 1981, CHI’86
In the early 80’s, Furnas, at Bell Labs, developed generalized fisheye views
A distortion technique that uses a Degree of Interest function to determine what information is shown and what is not shown
This example is a Fisheye Calendar focused around a particular date

11. SemNet - 1986 Fairchild, Poltrock, & Furnas, MCC
Semantic network 3D visualization
In the mid-80’s, Fairchild, Poltrock & Furnas at MCC developed SemNet
A visualization of a semantic network in 3D
Made use of fisheye views to manage complexity
First 3D network visualization

12. Cone Tree - 1991 Robertson, Mackinlay & Card, Xerox PARC, CHI’91
Limits:
10 levels
1000 nodes
Up to 10,000
At CHI’91, I introduced the Cone Tree hierarchy visualization
Work done with Jock Mackinlay and Stu Card at Xerox PARC
First 3D hierarchy visualization
Solved the loss of context problem with a focus and context display
Selected node and path to root rotate around to be shown in front
So, the context is always easily visible
Problems
Limits on scale: 10 levels and 1000 nodes for most efficient view
But can work up to 10,000 nodes
Occlusion forces user to select and animate (essentially a different form of scrolling)
Many others have proposed variants that solve these problems to one degree or another

13. Cone Tree – 10,000 nodes Performance problems and Visual clutter
This is what the Cone Tree looks like for 10,000 nodes (directories in a file system)
Given the hardware performance at the time, this was too slow

14. Fisheye Cone Tree Adding Fisheye solves problems
Speed problem was addressed simply by adding Fisheye view to Cone Tree
The Degree of Interest function took into account recent selections as well as the current selection

15. TreeMap - 1991 Johnson & Shneiderman, U. Maryland, Vis’91
Space filling
~3000 objects
MicroLogic’s DiskMapper
At about the same time, Johnson and Shneiderman at Univ. of Maryland developed Treemaps
Space filling hierarchy visualization that
uses containment to represent levels of the hierarchy
Deals with about 3000 visible nodes
Very good at revealing some properties of the nodes
For example, amount of disk space mapped to size of box
Also used color to encode additional property (e.g., age)
Less good at revealing structural relationships
This image is from DiskMapper, a commercial product from MicroLogic

16. FSN - 1992 Tesler, SGI, 1992 2D layout on 3D surface
Loss of context when examining detail
Hyper-G (Graz; Andrews; InfoVis’95)
In 1992, Joel Tesler at SGI introduced FSN
File System Navigator
Information Landscape
A 2D tree layout on the ground of a 3D environment
Selecting a node flies user to that node
Resulting in loss of context
Did have a second window with an overview
In 1995, Andrews at Univ. of Graz adapted FSN for web viewing in a system called Hyper-G

17. Graphical Fisheye - 1992 Sarkar & Brown, DEC SRC, CHI’92
Also a number of 2D network visualizations
Graphical Fisheye by Sarkar & Brown at DEC SRC in 1992
Generalization of Furnas fisheye view to 2D graph layout
Here the view on the right is the same structure, but focusing on node 48

18. Rubber Sheet - 1993 Sarkar & Snibbe, Brown, UIST’93
Rubber Sheet by Sarkar and Snibbe at Brown in 1993
Generalization of Graphical Fisheye to allow multiple focus

19. Spiral Visualization - 1994
Mackinlay, Robertson, & DeLine, Xerox PARC, UIST’94
Large DAGs
Only shows current path
Spiral Visualization, by Mackinlay, Robertson & DeLine at PARC
Enormous hierarchies and DAGs
Calendar and decision tree
Note that only the current path through the DAG is shown

20. Hyperbolic Browser - 1994 Lamping & Rao, Xerox PARC, UIST’94
Projected onto circle
1000’s of nodes
Hyperbolic Browser in 1994 by Lamping & Rao at PARC
A reaction to occlusion problems in the Cone Tree
Use of hyperbolic space instead of 3D perspective
Hyperbolic layout projected onto 2D circle
Can display 1000’s of nodes, with details diminishing as you near the edge
Has property that much information can be packed into the periphery

21. Continuous Zoom - 1995 Bartram et al, Simon Fraser, UIST’95)
Continuous Zoom, by Bartram et al at Simon Fraser Univ. in 1995
Generalization of Rubber Sheet visualization to allow smooth transitions while changing one or more foci of attention

22. Fsviz - 1995 Carriere and Kazman, Waterloo, InfoVis’95
Up to nodes with no occlusion
Fsviz, by Carriere and Kazman at Waterloo Univ. in 1995
Addressed occlusion problem in Cone Tree by moving to different viewpoint
Can layout 5000 nodes with no occlusion
But, they are small and far away

23. Butterfly - 1995 Mackinlay, Xerox PARC, CHI’95
Butterfly, by Mackinlay at PARC in 1995
citation index
Enormous DAG
Deals with asynchronous searches
Structure evolves

24. Index View - 1995 T. Masui, Sharp, UIST95 Vertical picks focus
Horizontal controls zoom
Aaaamon.dll
Edb500.dll
Label.exe
Oakley.dll
Tapi.dll
Wavemsp.dll
Ksproxy.ax
Ksqmf.ax
Ksuser.dll
Ksvpintf.ax
Lanman.drv
Legacy.inf
Linkinfo.dill
Lmrt.dll
Aaaamon.dll
Edb500.dll
Label.exe
Oakley.dll
Tapi.dll
Wavemsp.dll
Ksproxy.ax
Ksuser.dll
Legacy.inf
Lmrt.dll
Aaaamon.dll
Edb500.dll
Label.exe
Oakley.dll
Tapi.dll
Wavemsp.dll
Ksproxy.ax
Lmrt.dll
Aaaamon.dll
Edb500.dll
Label.exe
Oakley.dll
Tapi.dll
Wavemsp.dll
Index View, by Masui at Sharp in 1995
Primarily a list viewer for a small space, but can also be used for hierarchies
User moves mouse vertically to pick focus
Then horizontal to control zoom level
Does have a discoverability problem
So only works well if user knows what they are looking for
Partially addressed by using a permuted index

25. H3 - 1997 Munzner, Stanford Univ., InfoVis’97 Projected onto sphere
20,000 nodes
H3 layout algorithm, by Munzner at Stanford in 1997
An attempt to merge ideas from Hyperbolic browsers and Cone Trees
Projection is onto the surface of a sphere with the focus node at the center of the sphere
Able to display 20,000 nodes

26. Reconfigurable Disk Tree - 1998
Jeong and Pang, UC Santa Cruz, InfoVis’98
Number of identifiable nodes: – 4000
Reconfigurable Disk Trees, by Jeong and Pang at UC Santa Cruz in 1998
Another attempt to get around occlusion problems in Cone Tree
Both by using a different view point
And by collapsing the cones into disks
Displays 3000 – 4000 identifiable nodes

27. Disk Tree - 1998 Chi et al, Xerox PARC, CHI’98
Compact 2D representation
Disk Tree, by Chi et al at PARC in 1998
Compact 2D representation uses concentric circles, with a hierarchy level for each successive circle
Used to show structure of a web site
Line size and brightness encode page access frequency
Color encodes page lifecycle stage
Advantages
No occlusion
Compact
Pattern can be compared over time to observe changes

28. Sunburst - 2000 Stasko & Zhang, Georgia Tech, InfoVis 2000
Radial space-filling
Techniques for viewing more detail
Sunburst visualization, by John Stasko and Eugene Zhang at Georgia Tech, presented at InfoVis 2000
Basic Sunburst is similar to Disk Tree, except each level is space-filling
Study compared Sunburst to Treemap
Subjects preferred Sunburst because of more explicit representation of structure
Focus of InfoVis work was on new techniques to show additional detail
Angular detail of a focus region
“Detail outside” shrinks context to inner space and expands focus region to outer circles
“Detail inside” is opposite

29. Visualization Taxonomy - 1994
Noik (Graphics Interface’94)
Implicit (use of perspective)
Continuous focus and context
Filtered (removing items of low interest)
Discrete focus and context
Distorted (size, shape, position of elements)
Adorned (color, texture)
- Noik’s taxonomy sheds light on what these various techniques are doing
- Implicit (use of perspective)
- Advantage: continuous focus and context
- Filtered (removing items of low interest)
- Fisheye and dynamic queries
- Disadvantage: discrete focus and context
- Distorted (size, shape, position of elements)
- Fisheye also does this
- Can make it hard for user to track items of interest
- Adorned (color, texture)

30. Unresolved Problems Multiple focus
We completed a whirlwind survey of many hierarchy visualization techniques developed over the last 20 years
Each solves some of the problems we originally observed
None solve all the problems, and there are still some problems that need more work
For example, multiple focus has been addressed by only a few of the techniques
Yet, in real use, this is one of the biggest problems
Here are some mockups of some ideas we have been playing with

31. Folding Paper - proposal
This uses a metaphor of folding paper
Areas of interest are stretched out
Other areas are stacked
A focus area can have a second view that is magnified to reveal some detail

32. Multiple Focus in 3D - proposal
This is a similar idea, but uses 3D to display a series of columns, one for each level
When there are areas of multiple focus, more than one list will appear in a column
3D is used only to reduce screen space, not to encode any properties of the hierarchy

33. InfoBowl - proposal
The InfoBowl is a suggestion for how to introduce multiple focus into hyperbolic browsers
Each focus has a second level hyperbolic browser for exploring the next level of detail
Note: These are meant to be brainstorming examples. The hope is that more people will explore the problem of multiple focus in hierarchy visualizations

34. Unresolved Problems Multiple hierarchies Multiple focus
Almost all visualization techniques so far have shown only one hierarchy
But, we are beginning to see examples where the user must deal with and understand multiple hierarchies

35. Multiple Hierarchies One hierarchy changing over time Time Tube
Taxonomy visualization
MultiTrees (shared subtrees)
XML3D
Polyarchy (multiple intersecting hierarchies)
There are at least three kinds of multiple hierarchy problem
Two recent projects have explored what it means to view a hierarchy that changes over time
In essence, these are looking at multiple hierarchies that are very similar
MultiTrees are multiple hierarchies with shared subtrees
The third kind, which we call Polyarchies, are multiple intersecting hierarchies

36. Time Tube - 1998 Chi et al., Xerox PARC, CHI’98
In 1998, Chi et al at PARC introduced the Time Tube
A sequence of Disk Trees used to visualize changes in a web site over time
This is an example of viewing multiple similar hierarchies

37. Taxonomy Visualization - 2000
Graham et al., Napier Univ. IJHCS 2000
Recently, Graham et al at Napier Univ. explored visualizing changing biological taxonomies
In this example, individual objects are tracked across the hierarchies
Graham told me informally that this approach was abandoned when they tried to scale it up to real sized problems

38. MultiTrees - 1994 Furnas & Zacks, Bell Core, CHI’94
DAG’s with shared subtrees
MultiTrees were introduced by Furnas and Zacks at Bell Core in 1994
They are basically large DAG’s with shared subtrees
The typical example is about a class syllabus created by one professor
A second professor reuses some of the material, but only a subset
A third professor uses a different subset, possibly reusing some of the material introduced by the second professor

39. XML3D - 2000 Munzner, Stanford, IJHCS 2000
Recently, Munzner at Stanford used the H3 hyperbolic projection onto a sphere
To encode information from XML structures with multiple inheritance
This is essentially a MultiTree problem
User study
12,000 node topic hierarchy (Snap.com)
XML3D vs web-based solution vs Windows tree control
XML3D was reliably faster than the two 2D solutions

40. Polyarchy Visualization Problem: People and Resources Example
Multiple Hierarchies Exist
Direct reporting
Cost or Profit Center
Location
Implicit relationships
MultiTrees involve multiple hierarchies pointing to the same set of leaves (I.e., shared subtrees)
Polyarchies are about multiple intersecting hierarchies (a superset of MultiTrees)
He are some examples from a particular problem we are currently working on.
Problem comes from Metadirectory Services product
How do you bring together various information sources in an enterprise and provide a common access point?
There will be multiple hierarchies and other relationships
In this example, we are looking at information about people from four different information sources
In spite of having multiple hierarchies, the current solution one shows one hierarchy
But only one hierarchy is shown

41. Show Other Hierarchies
Find:
Daniel Robbins
Naming hierarchy is default
Allow maintainer to define hierarchies
Show search results in current view
Morph between views
There is a default hierarchy, called the naming hierarchy
We give the metaverse designer a way to specify what hierarchies should be exposed to the user
Search results are always displayed in the currently viewed hierarchy
We morph between views – we call this a visual pivot

42. Visualization Opportunities
Perceptual cues to provide pattern information
User can search or browse
Show hierarchies an object participates in
Show relationships between hierarchies
Show relationships between objects
Given the existence of multiple intersecting hierarchies and other relationships
How do we display this complexity to the user in a way that makes sense?
This is work we are currently engaged in
In particular, we want to show the hierarchies an object participates in
And the relationships between those hierarchies
As well as the relationships between the objects in the hierarchies

43. One Selection, One Hierarchy
Here is what the current prototype looks like
Upper left: user enters search terms and identifies their type
Lower left: results of search are displayed
Upper right: user picks a dimension (a hierarchy or relationship)
Lower right: the current selected people are displayed in the current selected dimension
In this case, Anoop Gupta is shown in the management hierarchy

44. Visual Pivot – Other Hierarchies
The user can pivot to any other dimension that the selected node participates in
This movie shows the original visual pivot animation
First pivoting to the business unit hierarchy
Then, related people
Finally, office location

45. Relationships Between People
The Related People dimension is particularly interesting
It shows a complex relationship
Management chain
Direct reports
Immediate peers
Some dimensions are simple, others quite complex

46. Relationships Between Multiple People
Often, users want to see how various people relate to each other
The interface allows the user to add as many people to the display as desired
Here we see two people in the management hierarchy
It finds the manager they have in common and displays both paths
Notice in all of these displays, we are managing an enormous database by
showing the minimum set of information that meets the user’s needs
Allowing the user to easily change the information shown

47. Unresolved Problems Multiple focus Multiple hierarchies
Evaluate hierarchy visualizations
Another key unresolved problem is that we must evaluate these techniques
Identify problems that can be resolved in later versions
Demonstrate the actual value for an end user
Basis for economic and managerial decisions for investment in this technology

48. Polyarchy Visualization User Studies
Study 1: Mockup of visual pivot
Issues list guided development of prototype
Study 2: Prototype: 2D vs 3D
Visual Pivot animation was misleading
Animation Speeds were too slow
To illustrate, let’s look at the studies done on the Polyarchy Visualization
We have done 4 user studies so far
First, we built a mockup of the visual pivot and did a simple usability study to determine if users could make sense of the complexity
Users had no difficulty in understanding what visual pivot was about
Users had high satisfaction ratings of the 3D visual pivot
The issues list then guided the development of a working prototype
Second, we did a usability study of a working prototype
We hoped to get performance data comparing 2D and 3D solutions
But, there were usability issues that still needed to be resolved
The visual pivot animation caused some problems because of poor readability of text during the pivot
The animation speeds were clearly too slow

49. Polyarchy Visualization User Studies
Study 3: Animation Styles and Speeds
Six animation styles: Picked two best
Twice as fast as study 2: Still too slow
Study 4: Prototype: 2D vs 3D
Identified most effective animation style
Identified best speed range
The third study was specifically designed to evaluate six alternative animation styles and to identify the appropriate speed range
The most recent study was another performance study, similar to Study 2 except that the usability issues were resolved

50. Two Styles of Visual Pivot
Sliding
Rotating
The results of the third study indicated that two particular pivot styles should be further evaluated
Initial pivots had a final step that moved to a preferred position and scale
The study showed that people had trouble understanding that because the new information was not always readily visible
The revised animations do that movement first rather than last
The study also showed that users had difficulty reading text while rotating around the vertical axis
Revised pivot is around the horizontal axis
A new pivot was added that slides the new information from right to left, then up a small amount
This allows reading the text of old and new side by side

51. Visual Pivot Styles Sliding versus Rotating
The fourth study tested these styles
Looking at performance data, there is a reliable advantage to the sliding pivot
Satisfaction data was mixed, with no clear preference

52. Visual Pivot Styles Learning Effects
Performance data shows a reliable learning effect,
with the sliding pivot easier to learn

53. Visual Pivot Speed Pivot times are broken into three phases:
Reposition
Pause after new information is shown
Pivot
Total Times tested: 0 sec, ½ sec, 1 sec, 2 sec
User task completion time was reliably slower with the slow animation
But, the other animation speeds, as well as no animation, were not reliably different
Can conclude that 3D animation does not incur any performance penalty
Preference data indicated a reliable preference for the “fast” animation
We are continuing the study of these visual pivot techniques

54. Unresolved Problems Get the solutions out to real users!
Multiple focus
Multiple hierarchies
Evaluate hierarchy visualizations
Get the solutions out to real users!
And finally, we need to get hierarchy visualization solutions demonstrated to be of value out into the mass market

55. Summary Use perception to reduce cognitive overhead
Need continued research to solve problems
Help user focus on multiple items of interest without losing context
Multiple hierarchies and other relationships
Handle larger scale
Evaluate  Iterate  get results out
In summary
We need to continue exploring ways to use perception to reduce cognitive overhead
We need to continue research into novel hierarchy visualizations
We need better ways to help the user focus on multiple items of interest without losing context
We need new solutions for multiple hierarchy and polyarchy visualization
Handle larger scale – 100,000’s to millions
And finally, we need more user studies to evaluate what we are doing
This should lead to better designs
And is ultimately a requirement for getting our results out to the mass market