1. EvaluationIntroVis/GfxInteractionWrap-up Thinking Interactively with Visualizations Remco Chang UNC Charlotte Charlotte Visualization Center

2. EvaluationIntroVis/GfxInteractionWrap-up Definition of Visual Analytics Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces [Thomas & Cook 2005] Since 2004, the field has grown significantly. Aside from tens to hundreds of domestic and international partners, it now has a IEEE conference (IEEE VAST), an NSF program (FODAVA), and a forthcoming IEEE Transactions journal.

3. EvaluationIntroVis/GfxInteractionWrap-up Individually Not Unique Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Data Mining Machine Learning Databases Information Retrieval etc Tech Transfer Report Generation etc Quality Assurance User studies (HCI) etc Interaction Design Cognitive Psychology Intelligence Analysis etc. InfoVis SciVis Graphics etc

4. EvaluationIntroVis/GfxInteractionWrap-up In Combinations of 2 or 3… Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Data Mining Machine Learning Databases Information Retrieval etc InfoVis SciVis Graphics etc

5. EvaluationIntroVis/GfxInteractionWrap-up In Combinations of 2 or 3… Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Interaction Design Cognitive Psychology Intelligence Analysis etc. Tech Transfer Report Generation etc

6. EvaluationIntroVis/GfxInteractionWrap-up This Talk Focuses On… Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Interaction Design Cognitive Psychology Intelligence Analysis etc. InfoVis SciVis Graphics etc Quality Assurance User studies (HCI) etc

7. EvaluationIntroVis/GfxInteractionWrap-up Interactive Analysis + Visualization Most people in the visualization community believe that interactivity is essential for visualization and visual analytics: – “A [visual] analysis session is more of a dialog between the analyst and the data… the manifestation of this dialog is the analyst’s interactions with the data representation” [Thomas & Cook 2005] – “Without interaction, [a visualization] technique or system becomes a static image or autonomously animated images” [Yi et al. 2007] The goal of this talk is to consider the role of interaction in computer graphics, information visualization, and visual analytics. First, we consider a stereotypical graphics application and try adding interaction to it..

8. EvaluationIntroVis/GfxInteractionWrap-up Urban Simplification (left) Original model, 285k polygons (center) e=100, 129k polygons (45% of original) (right) e=1000, 53k polygons (18% of original) R. Chang et al., Legible simplification of textured urban models. IEEE Computer Graphics and Applications, 28(3):27–36, 2008. R. Chang et al., Hierarchical simplification of city models to maintain urban legibility. ACM SIGGRAPH 2006 Sketches, page 130, 2006.

9. EvaluationIntroVis/GfxInteractionWrap-up Urban Simplification Which polygons to remove? Original ModelSimplified Model using QSlim Our Textured ModelOur Model Visually different, but quantitatively similar!

10. EvaluationIntroVis/GfxInteractionWrap-up Urban Simplification The goal is to retain the “Image of the City” Based on Kevin Lynch’s concept of “Urban Legibility” [1960] – Paths: highways, railroads – Edges: shorelines, boundaries – Districts: industrial, historic – Nodes: Time Square in NYC – Landmarks: Empire State building

11. EvaluationIntroVis/GfxInteractionWrap-up Urban Visualization with Semantics How do people think about a city? – Describe New York… Response 1: “New York is large, compact, and crowded.” Response 2: “The area where I live there has a strong mix of ethnicities.” Geometric,Information,View Dependent (Cognitive)

12. EvaluationIntroVis/GfxInteractionWrap-up Urban Visualization Geometric – Create a hierarchy of shapes based on the rules of legibility Information – Matrix view and Parallel Coordinates show relationships between clusters and dimensions View Dependence (Cognitive) – Uses interaction to alter the position of focus R. Chang et al., Legible cities: Focus-dependent multi-resolution visualization of urban relationships. IEEE Transactions on Visualization and Graphics, 13(6):1169–1175, 2007

13. EvaluationIntroVis/GfxInteractionWrap-up The Role of Interaction in Visualization We can use interactions to… [Yi et al. 2007] – Select: mark something as interesting – Explore: show me something else – Reconfigure: show me a different arrangement – Encode: show me a different representation – Abstract/Elaborate: show me more or less detail – Filter: show me something conditionally – Connect: show me related items In other words, we can use interactions to think.

14. EvaluationIntroVis/GfxInteractionWrap-up (1) WireVis: Financial Fraud Analysis In collaboration with Bank of America – Looks for suspicious wire transactions – Currently beta-deployed at WireWatch – Visualizes 15 million transactions over 1 year Uses interaction to coordinate four perspectives: – Keywords to Accounts – Keywords to Keywords – Keywords/Accounts over Time – Account similarities (search by example)

15. EvaluationIntroVis/GfxInteractionWrap-up (1) WireVis: Financial Fraud Analysis Heatmap View (Accounts to Keywords Relationship) Strings and Beads (Relationships over Time) Search by Example (Find Similar Accounts) Keyword Network (Keyword Relationships) R. Chang et al., Scalable and interactive visual analysis of financial wire transactions for fraud detection. Information Visualization,2008. R. Chang et al., Wirevis: Visualization of categorical, time-varying data from financial transactions. IEEE VAST, 2007.

16. EvaluationIntroVis/GfxInteractionWrap-up (2) Investigative GTD Collaboration with U. Maryland’s DHS Center of Excellence START (Study of Terrorism And Response to Terrorism) – Global Terrorism Database (GTD) – International terrorism activities from 1970-1997 – 60,000 incidents recorded over 120 dimensions Visualization is designed to be “investigative” in that it is modeled after the 5 W’s: – Who, what, where, when, and [why] – Interaction allows the user to adjust one or more of the W’s and see how that affects the other W’s

17. EvaluationIntroVis/GfxInteractionWrap-up (2) Investigative GTD Where When Who What Original Data Evidence Box R. Chang et al., Investigative Visual Analysis of Global Terrorism, Journal of Computer Graphics Forum (Eurovis), 2008.

18. EvaluationIntroVis/GfxInteractionWrap-up WHY ? WHY ? This group’s attacks are not bounded by geo-locations but instead, religious beliefs. Its attack patterns changed with its developments. (2) Investigative GTD: Revealing Global Strategy

19. EvaluationIntroVis/GfxInteractionWrap-up Domestic Group A geographically- bounded entity in the Philippines. The ThemeRiver shows its rise and fall as an entity and its modus operandi. (2) Investigative GTD: Discovering Unexpected Temporal Pattern

20. EvaluationIntroVis/GfxInteractionWrap-up (3) iPCA: Interactive PCA Quick Refresher of Principle Component Analysis (PCA) – Find most dominant eigenvectors as principle components – Data points are re-projected into the new coordinate system For reducing dimensionality For finding clusters For many (especially novices), PCA is easy to understand mathematically, but difficult to understand “semantically”.

21. EvaluationIntroVis/GfxInteractionWrap-up (3) iPCA: Interactive PCA R. Chang et al., iPCA: An Interactive System for PCA-based Visual Analytics. Computer Graphics Forum (Eurovis), 2009. To Appear.

22. EvaluationIntroVis/GfxInteractionWrap-up (3) Evaluation – iPCA vs. SAS/INSIGHT Results – A bit more accurate – People don’t “give up” – Not faster Overall preference – Using letter grades (A through F) with “A” representing excellent and F a failing grade.

23. EvaluationIntroVis/GfxInteractionWrap-up If (Interactions == Thinking)… What is in a user’s interactions? If (interactions == thinking), what can we learn from the user’s interactions? Is it possible to extract “thinking” from “interactions”?

24. EvaluationIntroVis/GfxInteractionWrap-up What is in a User’s Interactions? Types of Human-Visualization Interactions – Word editing (input heavy, little output) – Browsing, watching a movie (output heavy, little input) – Visual Analysis (closer to 50-50) VisualizationHuman Output Input Keyboard, Mouse, etc Images (monitor)

25. EvaluationIntroVis/GfxInteractionWrap-up What is in a User’s Interactions? Goal: determine if there really is “thinking” in a user’s interactions. Analysts Grad Students (Coders) Logged (semantic) Interactions Compare! (manually) Strategies Methods Findings Guesses of Analysts’ thinking WireVis Interaction-Log Vis

26. EvaluationIntroVis/GfxInteractionWrap-up What’s in a User’s Interactions From this experiment, we find that interactions contains at least: – 60% of the (high level) strategies – 60% of the (mid level) methods – 79% of the (low level) findings R. Chang et al., Recovering Reasoning Process From User Interactions. IEEE Computer Graphics and Applications, 2009. R. Chang et al., Evaluating the Relationship Between User Interaction and Financial Visual Analysis. IEEE Symposium on VAST, 2009.

27. EvaluationIntroVis/GfxInteractionWrap-up What’s in a User’s Interactions Why are these so much lower than others? – (recovering “methods” at about 15%) Only capturing a user’s interaction in this case is insufficient.

28. EvaluationIntroVis/GfxInteractionWrap-up Lessons Learned We have proven that a great deal of an analyst’s “thinking” in using a visualization is capturable and extractable. Using semantic interaction capturing, we might be able to collect all the thinking of expert analysts and create a knowledge database that is useful for – Training: many domain specific analytics tasks are difficult to teach – Guidance: use existing knowledge to guide future analyses – Verification, and validation: go back and check to see if everything was done right. But not all visualizations are interactive, and not all thinking is reflected in the interactions. – A model of how and what to capture in a visualization for extracting an analyst’s thinking process is necessary.

29. EvaluationIntroVis/GfxInteractionWrap-up Conclusion Interactions are important for visualization and visual analysis – In considering interactions, one must be aware of the necessary speed and frame rate of the displays. Techniques such as simplification, LOD, or approximation can be used. – Interactions have been proven to help the understanding of complex problems. Relevant interactions have been integrated in multiple visualizations for different domains and demonstrated significant impact. – Capturing and storing analysts’ interactions have great potential They can be aggregated to become a “knowledge database” that has traditionally been difficult to create manually.

30. EvaluationIntroVis/GfxInteractionWrap-up Discussion What interactivity is not good for: – Presentation – YMMV = “your mileage may vary” Reproducibility: Users behave differently each time. Evaluation is difficult due to opportunistic discoveries.. – Often sacrifices accuracy iPCA – SVD takes time on large datasets, use iterative approximation algorithms such as onlineSVD. WireVis – Clustering of large datasets is slow. Either pre-compute or use more trivial “binning” methods.

31. EvaluationIntroVis/GfxInteractionWrap-up Discussion Interestingly, – It doesn’t save you time… – And it doesn’t make a user more accurate in performing a task. However, there are empirical evidence that using interactivity: – Users are more engaged (don’t give up) – Users prefer these systems over static (query-based) systems – Users have a faster learning curve We need better measurements to determine the “benefits of interactivity”

32. EvaluationIntroVis/GfxInteractionWrap-up Future Work Analytical Reasoning and Interaction Visual Representation Production, Presentation Dissemination Data Representation Transformation Validation and Evaluation Data Mining Machine Learning Databases Information Retrieval etc Tech Transfer Report Generation etc Quality Assurance User studies (HCI) etc Interaction Design Cognitive Psychology Intelligence Analysis etc. InfoVis SciVis Graphics etc

33. EvaluationIntroVis/GfxInteractionWrap-up Future Work Lots of possible combinations. Are they all meaningful? Of particular interest to me is “Data + Interaction + Visualization” – How to apply computational approaches to find solutions that are usable by humans? Linear (PCA) and non-linear (manifold learning) create dimensions that are semantically difficult to define Nodes within a Bayesian network are difficult to comprehend, therefore the results difficult to take at face value.

34. EvaluationIntroVis/GfxInteractionWrap-up Thank you! rchang@uncc.edu http://www.viscenter.uncc.edu/~rchang