1. Opening the Black Box of Interaction in Visualization Hans-Jörg Schulz 1, Tatiana v. Landesberger 2, Dominikus Baur 3 1.Fraunhofer IGD, Rostock, Germany 2.TU Darmstadt, Darmstadt, Germany 3.Dominikus Baur Interfacery VIS Tutorial 2014 D.MINIK.US

2. 2VIS Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur

3. Part 1: Interaction activities 3Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur   Activities: What the user does to trigger a change in the computer (Action) Metaphor: What the user thinks the computer is doing and vice versa (Understanding) Architecture: What the computer actually does (Reaction)  

4. Part 1: Interaction activities 4Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur   Activities: What the user does to trigger a change in the computer (Action) Metaphor: What the user thinks the computer is doing and vice versa (Understanding) Architecture: What the computer actually does (Reaction)  

5. Overview of Part 1 6W’s of User’s Interaction and interaction loop Systematization of interaction Human (Ws) and System (Vis) Vis/VA-focused systematizations: 1. Visualization, 2. Visual Data Mining 3. Reasoning Third view: Interaction Support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 5      

6. Motivation System developers: What to include in my system Researchers: What is there and what is missing Users: What to expect from the system Developers, researchers,…: Canonicum for evaluation and system testing Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 6

7. Motivation Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 7 Make it easier for you… Systematization of perspectives …

8. What is interaction from user’s point of view? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 8 http://www.smartgirl.org/brain-food/career-hubs/human-computer-interaction.png

9. 6Ws of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 9 W HY do we interact? What is the goal? W HAT is the purpose? What is the intended effect of interaction? HO W do we interact? Which means do we use/have at disposal? W HO interacts? Who are the users interacting? What is their background? W HEN do we interact? When is interaction needed? W HERE is interaction used? Where users interact? Context of interaction Effects and means [adapted & merged Roth13, Jansen et al 13]

10. 6Ws of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 10 W HY do we interact? What is the goal? W HAT is the purpose? What is the intended effect of interaction? HO W do we interact? Which means do we use/have at disposal? W HO interacts? Who are the users interacting? What is their background? W HEN do we interact? When is interaction needed? W HERE is interaction used? Where users interact? Context of interaction Effects and means [adapted & merged Roth13, Jansen et al 13]

11. 6Ws of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 11 W HY do we interact? What is the goal? W HAT is the purpose? What is the intended effect of interaction? HO W do we interact? Which means do we use/have at disposal? W HO interacts? Who are the users interacting? What is their background? W HEN do we interact? When is interaction needed? W HERE is interaction used? Where users interact? Context of interaction Effects and means [adapted & merged Roth13, Jansen et al 13]

12. 6Ws of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 12 WHY do we interact? What is the goal? WHAT is the purpose? What is the intended effect of interaction? HOW do we interact? Which means do we use/have at disposal? WHO interacts? Who are the users interacting? What is their background? WHEN do we interact? When is interaction needed? WHERE is interaction used? Where users interact? [adapted & merged from Roth13, Jansen et al 13] Context of interaction Effects and means

13. Hierarchic View on Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 13 WHY do we interact? WHAT is the purpose? HOW do we interact?

14. Norman’s Model of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 14  [Norman88] 1. Establish a goal (Why?) 2. Form intention/identify task (What?) 3. Specify action sequence (How?) EXECUTION 7. Evaluate the outcome (Why?) 6. Interpret the system’s state (What?) 5. Perceive the state of the system (How?) EVALUATION  Execution/ Evaluation loop 4. Execute action 8. Take further action (compare outcome with goal)

15. Preliminary Summary 1.6Ws of interaction: 1.Effects+means: Why?, What? How? 2.Context: Who? Where? When? 2.Hierarchic nature of interaction 3.Execution/Evaluation loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 15

16. Which interactions exist? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 16

17. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 17

18. What is interaction  Systematization 1.6Ws of interaction: 1.Effects+means: Why?, What? How? 2.Context: Who? Where? When? 2.Hierarchic nature of interaction 3.Execution/Evaluation loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 18

19. What is interaction  Systematization 1.6Ws of interaction: 1.Effects+means: Why?, What? How? 2.Context: Who? Where? When? 2.Hierarchic nature of interaction 3.Execution/Perception loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 19  User-centric view  

20. What is interaction  Systematization 1.6Ws of interaction: 1.Effects+means: Why?, What? How? 2.Context: Who? Where? When? 2.Hierarchic nature of interaction 3.Execution/Perception loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 20  User-centric view   Visualization side?

21. Human ↔ Visualization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 21 WHY WHAT HOW   Subjective perception What should be modified in the view (goal/intention) How it should be modified (which action) Visualization changes What in the visualization is modified How this is done (software/hardware) [adjusted Roth13]

22. Human ↔ Visualization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 22 WHY WHAT HOW   [Card et al.99]

23. Human ↔ Visual Analytics Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 23 WHY WHAT HOW   Visualization Knowledge Models Data View manipulation Model visualizatio n Model buildin g Transformation Mapping Parameter refinemen t Feedback loop Information mining [Keim et al. 2008]

24. Information Visualization Model Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 24   [Card et al.99]

25. Information Visualization Model Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 25 Data Visual  [Card et al.99]

26. Visualization  Visual Analytics Simple Information Visualization Model Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 26 Visualization KnowledgeData View Manipulation Transformation Mapping  [Keim et al. 2008]

27. Visualization  Visual Analytics Simple Data Mining Model Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 27  Knowledge Data Mining/ Models Data Transformation Parameter refinement Information mining [Keim et al. 2008]

28. Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Feedback loop Information mining Visual Analytics Model Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 28  [Keim et al. 2008]

29. Way 1: InfoVis Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Feedback loop Information mining 3 Ways of Visual Analytics Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 29 

30. Way 2: Visual Data Mining Feedback loop 3 Ways of Visual Analytics Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 30 [Keim et al. 2008] Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Information mining 

31. Way 3: Provenance/Sensemaking/Reasoning Feedback loop 3 Ways of Visual Analytics Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 31 Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Information mining  [Keim et al. 2008]

32. Support all 3 ways via visual means Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Feedback loop Information mining Interaction Need in Visual Analytics Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 32  [Keim et al. 2008]

33. Systematization of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 33 WHY WHAT HOW  2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […] 1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 

34. Systematization of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 34 WHY WHAT HOW  2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […] 1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 

35. Levels of Systematization: Example Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 35 WHY WHAT HOW  [Yi et al 07] Select, Explore, Reconfigure, Encode, Abstract/Elaborate, Filter, Connect  ?

36. Levels of Systematization: Problem of ambiguous terms Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 36 WHY WHAT HOW  [Yi et al 07] Select, Explore, Reconfigure, Encode, Abstract/Elaborate, Filter, Connect  ? What/where in the pipeline?

37. Pipeline-Focused Interaction Systematization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 37 1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […]

38. Pipeline-Focused Interaction Systematization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 38 1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […] ?

39. Unified VA Interaction Systematization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 39 1. Visualization Data changes Selection Editing Visualization changes Scheme: type and mapping Parameters 2. Data mining Data changes Selection Editing Data mining changes Scheme: data processing type Parameters 3. Reasoning Data changes Analytic process tracking Editing (annotation) Reasoning changes Scheme: change analysis type Parameters [von Landesberger et al. 2014]

40. 3 Ways of Visual Analytics: InfoVis (Way 1) Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 40 Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Feedback loop Information mining 

41. 1. Visualization Systematizations Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 41 1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] Infovis pipeline based systematization

42. 1. Visualization Systematization #Words#Sentence 509 10020 807 Text #Words Size #Senten ces Color [Card et al 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur

43. #Words#Sentence 509 10020 807 Text #Words Size #Senten ces Color InfoVis Interaction: View Transformation [Card et al 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur

44. Navigation Pan, zoom, scroll,... Highlighting Hover Select+highlight Brushing and linking Magic lenses View reconfiguration (Re-)arrange multiple views on the screen Open/close new views Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 44 InfoVis Interaction: View Transformation Source: maps.google.com Navigation in visible space

45. Navigation Pan, zoom, scroll,... Highlighting Hover Select+highlight Brushing and linking Magic lenses View reconfiguration (Re-)arrange multiple views on the screen Open/close new views Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 45 InfoVis Interaction: View Transformation Source: maps.google.com Navigation in visible space NAVIGATE: Paris  Sydney Problem: Cumbersome, time consuming, “lost in space”

46. Video01: Topology-Aware Navigation in Large Networks Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 46 [Moskovich et al.09] InfoVis Interaction: View Transformation Navigation

47. Pan, zoom, scroll,... Highlighting Hover Select+highlight Brushing and linking Magic lenses View reconfiguration (Re-)arrange multiple views on the screen Open/close new s Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 47 InfoVis Interaction: View Transformation Highlighting important information

48. Navigation Pan, zoom, scroll,... Highlighting Hover Select+highlight Brushing and linking Magic lenses View reconfiguration (Re-)arrange multiple views on the screen Open/close new views Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 48 InfoVis Interaction: View Transformation Configuring multiple views

49. InfoVis Interaction: Visual Mapping Text #Words#Sentence 509 10020 807 Text #Words Size #Senten ces Color Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 49 [Card et al 99]

50. InfoVis Interaction: Visual Mapping Visualization type Type of visualization Scatterplot/matrix Node-link/matrix Type of mapping E.g. color/size/form Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 50 [van den Elzen & van Wijk 13] Video 10

51. InfoVis Interaction: Visual Mapping Visualization type Type of visualization Type of mapping Mapping parameter Data  mapping E.g. color scheme Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 51 [van den Elzen & van Wijk 13] Video 10

52. InfoVis Interaction: Visual Mapping Visualization type Type of visualization Type of mapping Mapping parameter Data  mapping E.g. color scheme Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 52 [van den Elzen & van Wijk 13] Video 10

53. InfoVis Interaction: Visual Mapping Visualization type Type of visualization Type of mapping Mapping parameter Data to be mapped E.g. color scheme Further specifics E.g. type of layout, sorting Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 53 [van den Elzen & van Wijk 13] Video 10

54. InfoVis Interaction: Data Manipulation Text #Words#Sentence 509 10020 807 #Words Size #Senten ces Color Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 54 [Card et al 99]

55. InfoVis Interaction: Data Manipulation Data navigation drill down, expand, filter, … Data transformation Change data values by calculation Normalization, aggregation,… Data editing Change data values by editing Create data Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 55

56. InfoVis Interaction: Data Manipulation Data navigation drill down, expand, filter, … Data transformation Change data values by calculation Normalization, aggregation,… Data editing Change data values by editing Create data Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 56 Top down Filter, details on demand Bottom up Expand on demand Middle out Start in the middle [von Landesberger et al 11]

57. InfoVis Interaction: Data Manipulation Data navigation drill down, expand, filter, … Data transformation Change data values by calculation Normalization, aggregation,… Data editing Change data values by editing Create data Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 57 Top down Filter, details on demand Bottom up Expand on demand Middle out Start in the middle [van Ham & Perer 09] Search, Show Context, Expand on Demand

58. InfoVis Interaction: Data Manipulation Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 58 Video 02: data search and expand

59. InfoVis Interaction: Data Manipulation Data navigation drill down, expand, filter, … Data transformation Normalization (lin, log, exp,..) Aggregation (manual, according to data,…) … Data editing Change data values by editing Create data Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 59 Source: Gapminder.org

60. InfoVis Interaction: Data Manipulation Data navigation drill down, expand, filter, … Data transformation Normalization (e.g. lin, log) Aggregation Manual According to data attributes According to data structure (e.g. communities) Etc. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 60

61. InfoVis Interaction: Data Manipulation Data navigation drill down, expand, filter, … Data transformation Normalization (lin, log, exp,..) Aggregation (manual, according to data,…) Data editing Change values Create data Individual values Whole datasets Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 61

62. InfoVis Interaction: Data editing Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 62 Video 03: edit

63. InfoVis Interaction: Data creation Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 63 [Bremm et al 2012] PC/DC On the Highway to Data [Bremm et al 2012] Video 04: create

64. Summary: InfoVis Interaction #Words#Sentence 509 10020 807 Text #Words Size #Senten ces Color

65. Way 2: Visual data mining Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 65 Visualizatio n Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Information mining 

66. VA Interaction Systematization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 66 2. Visual Data Mining [Bertini & Lalanne 09] […] 1. Information Visualization

67. Visual Data Mining Computationally enhanced Visualization (V++) Visually enhanced Mining (M++) Integrated Visualization and Mining (VM) Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 67 [Bertini & Lalanne09]

68. Visual Data Mining Interaction Manipulating and tuning: Changing the scheme: Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 68 Vis: changing representation parameters DM: changing model parameters [Bertini & Lalanne09] Vis: changing the visual mapping or visual representation DM: changing the data model

69. Visual Data Mining Interaction Manipulating and tuning: Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 69 Vis: changing representation parameters DM: changing model parameters For example: changing color schemeFor example which motif is searched

70. Visual Data Mining Interaction Changing the scheme : Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 70 For example: changing node-link diagram to adjacency matrix For example motif search vs clustering Vis: changing the visual mapping or visual representation DM: changing the data model

71. Visual Data Mining Interaction: Motif search and Visualization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 71 [von Landesberger et al 09] Video 05: data mining

72. Way 3: Feedback loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 72 Visualization Knowledge Models Data View manipulation Model visualization Model building Transformation Mapping Parameter refinement Information mining  [Keim et al. 2008]

73. VA Interaction Systematization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 73 2. Visual Data Mining 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […] 1. Information Visualization

74. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 74 [Heer & Shneiderman12] Reasoning/Provenance Systematization Visualization Provenance

75. 75 [Heer & Shneiderman12] Reasoning/Provenance Systematization Visualization Provenance Insight Annotate, Bookmark History Record Guide redo, undo, revisit Cooperate Share Combined [Gotz & Zhou, Kerren et al, Heer & Shneiderman12]

76. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 76 [von Landesberger et al 14] Visualization Visual history & annotation

77. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 77 [von Landesberger et al 14] Visualization Visual history & annotation

78. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 78 [von Landesberger et al 14] Visualization Visual history & annotation

79. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 79 [von Landesberger et al 14] Visualization Visual history & annotation

80. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 80 [von Landesberger et al 14] Visualization Visual history & annotation Same interaction type  aggregation

81. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 81 [von Landesberger et al 14] Visualization Visual history & annotation Same interaction type  aggregation using interaction systematization

82. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 82 [von Landesberger et al 14] Visualization Visual history & annotation Aggregated interaction

83. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 83 [von Landesberger et al 14] Visualization Visual history & annotation

84. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 84 [von Landesberger et al 14] Visualization Visual history & annotation “Go back” - review/revise

85. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 85 [von Landesberger et al 14] Visualization Visual history & annotation “Go back” - review/revise

86. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 86 [von Landesberger et al 14] Visualization Visual history & annotation Start a different exploration path

87. Reasoning/Provenance Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 87 [von Landesberger et al 14] Visualization Visual history & annotation Start a different exploration path

88. Systematization of Interaction Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 88 WHY WHAT HOW  1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […]

89. Systematization of Interaction - according to InfoVis Pipeline Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 89 WHY WHAT HOW  1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […]

90. InfoVis-Focused Systematization: Problem of ambiguous terms Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 90 WHY WHAT HOW ? Why? What for? How exactly? “Change mapping”

91. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 91

92. Interaction hierarchy Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 92 WHY WHAT HOW [Sedic & Parsons 10]

93. Interaction hierarchy Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 93 WHY WHAT HOW [Sedic & Parsons 10] Time & cognitive burden

94. Three Dimensions: Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 94   Visualization Human Interaction support

95. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 95 “manual” “supported” “smart” “data-driven” “DOI-based” “automatic” Fast No cognitive burden No control Slow/cumbersome High cognitive burden Full control

96. Interaction support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 96 “manual” “supported” “smart” “data-driven” “DOI-based” “automatic” Fast No cognitive burden No control Slow/cumbersome High cognitive burden Full control

97. Interaction support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 97 “manual” “supported” “smart” “data-driven” “DOI-based” “automatic” Supported Snap to grid Edgelens [Wong & Carpendale03] Video 06: edgelens

98. Interaction support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 98 “manual” “supported” “smart” “data-driven” “DOI-based” “automatic” Fast No cognitive burden No control Slow/cumbersome High cognitive burden Full control Data driven Topology-aware navigation Data-aware selection Video 07: data driven

99. Interaction support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 99 “manual” “supported” “smart” “DOI-based” Guidance “automatic” Fast No cognitive burden No control Slow/cumbersome High cognitive burden Full control Guidance Small multiples DOI-based exploration “data-driven” Video 08: guidance

100. Interaction support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 100 “manual” “supported” “smart” “DOI-based” “automatic” Fast No cognitive burden No control Slow/cumbersome High cognitive burden Full control Smart Tableau “show me” “data-driven” [Heer et al 08]

101. Interaction support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 101 “manual” “supported” “smart” “data-driven” “DOI-based” “automatic” Fast No cognitive burden No control Slow/cumbersome High cognitive burden Full control Automatic NodeTrix Video 09: automatic

102. Part 1: Interaction actions Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 102

103. Summary Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 103 WHY WHAT HOW  1. Information Visualization [Pike et al 09] [Wybrow et al 14] [Parsons & Sedig 14] [Jansen, Dragicevic 13] [Roth13] [Shneiderman 96] [Keim02] [Dix & Ellis98] [Spence07] [Zhou & Fesner98] […] [Yi et al. 07] 2. Visual Data Mining [Bertini & Lalanne 09] […] 3. Reasoning [Heer & Shneiderman12] [Gotz & Zhou 08] [Kerren & Schreiber12] […]

104. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur 104

105. Next: Interaction architecture 105Vis Tutorial: Opening the Black Box of Interaction in Visualization – H.-J. Schulz, T. v. Landesberger, D. Baur   Activities: What the user does to trigger a change in the computer (Action) Metaphor: What the user thinks the computer is doing and vice versa (Understanding) Architecture: What the computer actually does (Reaction)   Hans-Jörg Schulz