1. How Human Brain Understand Visualisation and Graph Visualisation Evaluation Kai Xu National ICT Australia Sydney, Australia

2. Part I: How Human Braid Understand Visualisation

3. Acknowledgement Most of the content can be found in the book “Information Visualization – Perception for Design”. It’s author Colin Ware at University of New Hampshire is a close friend of the information visualisation research group at School of IT.

4. Table of Contents The human visual perception system. Visual Attention and pre-attentive patterns.

5. Visual Perception System VisualisationEyesBrain Information flow

6. Visual Perception System Brain processing speed Task complexity

7. Visual Perception System Stage 1 Rapid parallel processing: billions of neurons; Extraction of orientation, texture, color, and motion features.

8. Visual Perception System Stage 2 Slower processing than stage 1; Detection of 2D patterns, contours and regions.

9. Visual Perception System Stage 3 Slow serial processing; Involve both working and long-term memory; Object identification and eye-hand coordination.

10. Visual Angle Visual angle: the angle subtended by an object at the eye of an observer. –A thumbnail held at arm length subtends about 1 degree of visual angle.

11. Human Visual Field Visual field of view when gazing straight ahead. –Can see slightly more than 180 degrees horizontally; –Much less angle vertically. The irregular boundary of left and right fields are caused by facial features such as nose. The dark grey area is the region of binocular overlap.

12. Visual Acuity If focus on the central point, every character is about equally distinct. This is because the visual acuity decreases quickly with the distance from fovea.

13. Acuity test Line test Points test Acuity distribution

14. Pixels and Brain Pixels 1 bp Small Screen Big Screen Brain pixelscreen pixel

15. Ultimate Display

16. Visual Attention and Pre-Attentive Patterns In the stage 1 of perception system, the whole visual field is processed in parallel and very fast; The information that can be captured in this stage are easily distinguished. Pre-attentive patterns (pop-out effects). Should be considered when designing visualisation. Some examples.

17. Color

18. Orientation

19. Motion

20. Size

21. Simple shading

22. Conjunction (does not pop out)

23. Compound features (do not pop out)

24. Surrounded colors do not pop out

25. Laws of pre attentive display Must stand out on some simple dimension –color, –simple shape = orientation, size –motion, –depth Conjunctions of pre-attentive dimensions do not always work.

26. Lessons for Information Visualisation Can be used for individual symbols or areas; Avoid possible negative effect: –Do not use large areas of strong color. Orthogonality: use a different channel for a different type of information. Example: Mapping high dimensional data to display variables. –Position (2) –Orientation (1) –Size (1) –Motion (2)++ –Blinking (1) –Color (3) –…–…

27. Part II: Graph Visualisation Evaluation

28. Table of Contents Quick review Introduction Evaluation of graph drawing aesthetics Evaluation of graph layout methods Evaluation of large graph visualization Conclusions

29. Review - Drawing conventions

30. Review - Graph Layout Important part of graph visualization Finds “good” positions for nodes and edges –To improve “graph readability”, i.e., facilitate people's understanding of the graph structure Review: Layout algorithm covered in the previous lectures: –Tree layout –Layered layout (Sugiyama method) –Force-directed layout (spring algorithm)

31. Tree Layered drawing –nodes are placed on horizontal layers Radial drawing –the layers are mapped to concentric circles HV drawing –places the edges horizontally or vertically Space-filling methods (Treemap) –Inclusion indicates parent- child relationship; –Improved space efficiency

32. Directed Graph Layered drawing (framework) 1.Cycle removal: if there is directed cycles, temporarily reverses the direction of some to make the graph acyclic; 2.Layer assignment: nodes are assigned to horizontal layers, and thus determines their y- coordinate; 3.Crossing reduction: within each layer the nodes are ordered to reduce the number of crossings; 4.Horizontal coordinate assignment: the x-coordinates of each vertex is determined.

33. Undirected Graph Force-directed methods –A graph is treated as a system of entities with force acting between them. –The algorithm seeks a configuration with locally minimal energy, i.e., a position for every entity such that the sum of the forces on each entity is zero. –Common example Spring embedder

34. Introduction What’s covered so far: –Various Graph layout algorithms This lecture: how these affect people's understanding of the graph. –Are they effective at all? –Which one is relatively more effective? Also: visualization of large graphs –Where the traditional aesthetics and layout algorithms do not really work

35. Graph Drawing Aesthetics Aesthetics are the graphic properties layout algorithm try to optimise. Crossings: –Minimization of the total crossing number Area –Minimization of drawing area –Only meaningful to some layout. Example, grid drawing with integer coordinates Aspect ratio –The ratio of the long and short edge length of its covering rectangle –Ideal case is to obtain any aspect ratio in a given range (so the drawing can fit into differently shaped screen space)

36. Graph Drawing Aesthetics Edge length (several variations): –minimization of the sum of the edge length; –minimization of the maximum edge length; –minimization of the variance of the edge length; –only meaningful to some layout algorithm. Bends (several variations): –minimization of the total number of bends; –minimization of maximum number of bends on an edge; –minimization of the variance of the number of bends on the edge; –trivially satisfied by straight-line drawing.

37. Graph Drawing Aesthetics Angular resolution: –maximization of the smallest angle; –especially relevant for straight-line drawing. Symmetry: –display the symmetries of the graph in drawing –reflective and rotational symmetry Orthogonality: –how well the edges are parallel to the axes, and how well the nodes match to a grid; Upward flow: –for directed graph only, –how well edges are pointing to a specified direction (usually upward);

38. Evaluation Measuring the performance of subjects (users) completing certain task(s). Graph-related tasks are used for graph visualization. –Example: find the shortest path between 2 nodes in a graph. Performance is usually measured by –Accuracy –Completion time

39. Graph-Related Task Performance There are many factors affect the performance The difficulty level of the task –Simple: find all the neighbors of a node –Hard: find all the nodes have graph distance 2 to two given nodes. Size of the graphs –Small –Large Subjects background –Whether they are familiar with graph visualization or not –Whether they are familiar with certain application domain (for domain- specific tasks). And many more … Should consider/control as many as possible when doing a test.

40. Table of Contents Introduction Evaluation of graph drawing aesthetics Evaluation of graph layout methods Evaluation of large graph visualization Conclusions

41. Do Aesthetics Affect Graph Readability ? Problem: Most aesthetics are proposed based on experience –Later becomes something the research community agree on Study: readability of abstract graph –Tasks are not domain specific Purchase, H.C. et al. (1996)

42. Dataset Three aesthetics are tested: 1.Minimizing edge crossings, 2.Minimizing bends, and 3.Showing symmetry. Two planar graphs are used –one with 16 nodes and 18 edges –the other with 16 nodes and 28 edges Nine drawings are produced for each graph, –with three levels (few, some and many) of bends, crossings, and symmetry respectively. To isolate the effect of each aesthetic, the drawings with different bend levels shows no crossings or symmetry. –the same for the two other aesthetics (manual layout).

43. Tasks: 1.Shortest path: the length of the shortest path between two nodes; 2.Connections between nodes: minimum number of nodes needs to be removed to disconnect two nodes; 3.Connections between subgraphs: minimum number of edges needs to be removed to disconnect two subgraphs. Results –Effective: increasing bends or crossings decreases readability; –Not clear: symmetry.

44. Caveats Dataset is fairly simple –Small planar graph The selected tasks are similar –All focus on path between nodes or subgraphs, –this hardly cover all the information a graph structure can possibly convey. It is possible that change in the dataset and/or tasks can alter the results

45. Which Aesthetic is the most important? The relative importance among aesthetics Including 5 aesthetics: 1.minimizing edge crossings, 2.minimizing bends, 3.symmetry. 4.minimum angle 5.orthogonality Purchase, H.C (1997)

46. Dataset (Similar to last work) Planar graph with 16 nodes and 28 edges is used 5 aesthetics and 10 drawings –2 for each aesthetics: representing a strong or weak presence. b: bends, c: crossings, m: minimal angle, o: Orthogonality, s: symmetry

47. Tasks (the same as the last work) 1.Shortest path: between two nodes; 2.Connections between nodes: number of nodes to disconnect two nodes; 3.Connections between subgraphs: number of nodes to disconnect two subgraphs. Results –Most important: reducing the number of crossing; –Less effective: minimizing the number of bends and maximizing symmetry; –Not obvious: maximizing the minimum angle and orthogonality.

48. Does Aesthetics Affect Cognitive Load? From a cognitive psychology angle Testing aesthetics that affect shortest path task performance: –Continuity (path bendiness): the angular deviation from a straight line. –Number of crossings and average crossing angles: the crossings on the shortest path, and the angle of crossing. –Number of branches: the number of edges connect to the nodes on the shortest path but not part of the path. –Shortest path length and total edge length Ware, C. et al. (2002)

49. Task –find the shortest path between 2 given nodes Dataset –180 drawings with per-defined parameters. –42 nodes in each graph, –2 examples

50. Results: Important: path continuity. Edge crossings –Neutral: the total number of edge crossings in the graph. –Important: those cross the shortest path. Important: the number of branches emanating from nodes on the path.

51. Table of Contents Introduction Evaluation of graph drawing aesthetics Evaluation of graph layout methods Evaluation of large graph visualization Conclusions

52. How People Read Graph? Eye movement: when people reading graph. –using eye-tracking device. Dataset: 3 small social networks. Drawings: –Four drawings each graph ; –two circular layout and two radial layout; –The pair of same layout: one has more crossings than the other.

53. Task: shortest path between nodes Results: –Graph layout can affect: slow down trigger extra eye movements. –Caused by edges: incident to nodes concerned, going toward to the target node alongside the paths. Huang, W., Eades, P. (2005)

54. Layout vs. Graph Readability Comparing different layout methods. A planar graph of 17 nodes and 29 edges. maximum node degree is 4: –So it is applicable to orthogonal drawing ; –A quite strong constraint. 3 layout algorithms: –Force-directed: 3 variations; –Planar orthogonal grid drawing: 2 variation; –Planar grid drawing: 3 variations.

55. Tasks: 1.Shortest path between two nodes; 2.Disconnect two nodes; 3.Disconnect two subgraphs. Results: One planar grid drawing method (SEIS) produced significantly more errors than the rest; For the rest, the average response times were not significantly different. –So there is not much difference between layout algorithms! Purchase, H.C. (1998)

56. Are the Drawings Actually Different? For one drawing method, there are usually many variations –How different are the drawings? But first, what are the measurements? A study to find good measurements for orthogonal drawing. What’s interesting: –The authors tried to find out the measurements through evaluation, rather than just pick some. Bridgeman, S.S. and Tamassia, R. (2001)

57. What difference needs to be measured? Rotation: the minimum angle needs to be rotated to make one drawing the same as the other; Ordering: which drawing is more similar to the original; Magnitude: one drawing is k times more similar to the original than the other drawing.

58. The tests: Rotation: –Left: original drawing; right: same drawing with different orientations. –The user chooses the one that looks most similar.

59. Ordering: –Left: original drawing; the other two: modified drawings; –The user chooses the more similar one.

60. Magnitude Measures response times: –Assumption: user will complete the task quicker if the drawings are similar. Left: original drawing; Right: modified drawing; Identify missing vertex in the right drawing: –Vertices have labels, otherwise the task is too difficult.

61. Evaluated Measures (a lot) Corresponding Objects. Point Set Selection. Drawing Alignment. Suitability for Ordering vs. Rotation and Ordering. Notation. Undirected Hausdorff Distance. Paired Hausdorff Distance. Average Distance. Nearest Neighbor between. Orthogonal Ordering. Ranking. Average Relative Distance λ-Matrix Nearest Neighbor Within ε-Clustering Separation-Based Clustering Shape. …

62. Results Rotation: absolute and relative point positions are important. Ordering: point positions are less significant. Magnitude: no clear measurement is found. To recognize the graph as similar, the most important are : –the perimeter of the drawing; –The position and shape of few key features. To find a specific change (magnitude) –The drawings need to look very much alike, or –Some other cues (color change, more distinctive vertex names, etc.) are needed to highlight the change.

63. Table of Contents Introduction Evaluation of graph drawing aesthetics Evaluation of graph layout methods Evaluation of large graph visualization Conclusions

64. Is Large Graph Readable? A graph with 3200 nodes. Laid out using force-directed method. The “readability” will be more or less the same with other layout methods. Many real-world networks are much larger. What can we do?

65. Clustered Graph Clustering can show the structure of a large graph. –The nodes are partitioned into clusters; –The connections between clusters reveal the underlying graph structure. Example: a multi-level 2.5D drawing for hierarchically-clustered graphs. Difficult to evaluate: –Tightly coupled with interaction.

66. Extending Layout to 3D Many 2D layout methods can be extended to 3D: Natural extension: –Force-directed method; –Orthogonal drawing. Layered tree drawing –cone tree Poly-plane drawing –Placing subtree on poly-planes in 3D Layered drawing of directed graphs –Placing nodes on the parallel circles on the surface of a cone or cylinder –Placing nodes on parallel layers in vertical planes (“walls”)

67. Does 3D Really Improve Readability? Comparing various approaches to visualize a graph in 2D and 3D 2D: orthographic (parallel) projection Static Perspective: perspective projection Stereo: shutter glasses Passive rotation: automatic Hand coupled rotation: mouse-controlled rotation; Head coupled perspective: head-controlled perspective; Combination of previous approach: stereo, head coupled perspective. “Stereo, head coupled perspective” setup

68. Task: whether two nodes are connected by a path of length 2. Dataset: randomly laid out graph. Ware, C., Franck, G. (1996)

69. Results: A static perspective is only slightly better than a 2D diagram; 3D motion and stereo viewing both help but not particularly important; –Both are more significant than stereo cues. Stereo viewing alone increases the understandable graph size by a factor of 1.6; Head coupling alone increases by a factor of 2.2; Combine the two (head-coupled stereo viewing) increases by a factor of 3;

70. What Else Can We Do With 3D? Improve graph readability with “geon diagram” Geon: simple 3D shape with color and texture. Geon diagram: –Nodes: geons; –Edges: connections between geons. –Attributes of nodes and edges: geon color and texture.

71. Experiment 1 –UML diagrams: geon vs. 2D; –The subjects were first shown a structure in either geon or UML form, and later asked to identify it in a series of diagrams. Results: –Geon diagrams have half the errors and significantly faster. –Geon diagrams can be recalled much more reliably.

72. Experiment 2: –Geon diagram vs. 2D geon Dataset and tasks are the same. Results: –Geon is much more accurate and reliable compare to 2D geon.

73. Motion Using the forth dimension — time — to improve graph readability Different from dynamic or time-series graph visualization –Graph is not changed –Improving rather than reducing readability: it is usually more difficult to understand the graph when it changes. C. Ware and R. Bobrow. 2004.

74. Motion highlighting Circular: –A circular motion around the center position Jolt: –Moves in pulses; –Similar to an object oscillating from being struck briefly. Crawl: –Animated sawtooth patterns radiating out from the selected node Expanding nodes: –Grow larger and smaller periodically

75. Comparing Different Motion Highlighting Experiment 1 –Task: whether there are two red nodes within two links from a circled node. –Also included “no highlighting” and “static highlighting”, i.e., the selected subgraph is marked with different color. Results: –Motion highlighting requires half times and more accurate comparing to “no highlighting”; –Static highlighting is the most accurate, but slower than motion highlighting.

76. Experiment 2: Node and Edge Motion Highlighting Tasks: –Are there at least two red nodes within two links of the specified subgraph? –Are there at least two red links within two links of the specified subgraph? Results: –Motion highlighting both nodes and links is as good as or better than separate highlighting of nodes and links; –The static highlighting: As good as the motion techniques for revealing links Not as good as circular motion for revealing nodes.

77. Experiment 3: Complex Pattern Task: –Identify a chain of three red nodes connected by blue edges. Results: –Motion highlighting had similar performance as static highlighting. –The reason may be the pattern was relatively easy to identify, because it always started at the ringed node.

78. Motion in Large Graph Comparing 4 highlighting methods: 1.No highlighting; 2.Static highlighting; 3.Motion highlighting; 4.Static and motion highlighting. Static highlighting: –increase node size; –Change edge color. Combined highlighting –Combine static highlighting with pulse highlighting; –The larger nodes and links are in pulse highlighting mode.

79. Dataset: –Graph of five sizes are used: 32, 100, 320, 1000, and 3200. Task 1: –If there was a red node within 2 links of a specified node. Results 1: –Motion highlighting is accurate even for the largest diagrams. –Without highlighting, error rates were high even for the smallest network. –Motion highlighting and static highlighting were equally effective.

80. Task 2: if 2 subgraphs had nodes in common. Highlighting methods: –One with static highlighting and one with motion highlighting, or –Two with different motion highlighting. Results 2: –Best performer: one with motion highlighting and the other with static highlighting. Ware, C., Bobrow, R. (2005)

81. Beyond Node-Link Diagram Matrix Representation A graph can be represented by a connectivity matrix. Advantage: –no edge crossing Disadvantage: –large empty space for spare graph

82. A comparison between node-link diagram and matrix representation Tasks: –estimating the number of nodes; –estimating the number of links; –finding the most connected node; –finding a node with a given label; –finding a link between two specified nodes; –finding a common neighbor between two specified nodes; –finding a path between two nodes. (more variety of tasks) Dataset –Random graph of size: 20, 50, and 100 nodes; –For each size, different link density: 0.2, 0.4 and 0.6. Ghoniem, M. et al. (2004)

83. Results: When graphs are bigger than 20 vertices, matrix outperforms node- link diagrams on most tasks. –Only path finding is consistently in favor of node-link diagrams. For small graphs: –Node-link diagrams are always more readable than matrices; For larger graphs: –Matrices are 30% more accurate; –Matrices have comparable or better answer time. For more complex tasks such as “path finding”, interaction is needed: –For example, displaying all the possible paths after selecting two nodes; –For matrix, path can be displayed by connecting cells using curves (mix matrix with node-link diagram).

84. Summary Effectiveness of aesthetics: –Most effective: edge crossings; –Less effective: bends and symmetry; –Not obvious: minimum angle and orthogonality. Effectiveness of layout: –No significant difference between force-directed, orthogonal and planar layout methods. Large graphs: –3D is only better when coupled with interactions (and stereo vision if possible). –The 3D Geon diagram works better than 2D version; –Motion highlighting is better than static highlighting in some tasks. –Matrix representation is better than node-link diagram except path finding.

85. Caveat These conclusion are only valid in their test setup. For instance, “different layout methods have little impact on graph readability” when –Graph is small and simple –For shortest path task only It is possible that change in any test conditions can alter the results.