Sunil Gahlawat, Lu Gan, Miya Sylvester, Yiran Wang USABILITY AND UTILITY OF TOPICLENS A VISUALIZATION SYSTEM FOR THE EXPLORATION OF TOPIC MODELS AND SOCIAL NETWORKS

Overview  Identifying and classifying usability defects, including overlooked problems in system.  Study whether the design elements and aesthetics enhance the Usability of the program.  Evaluate User satisfaction/User experience in terms of ease of learning, retention, efficiency, and reliability.

User Study Set-Up  Pre-Survey  Part 1 experiment (Editing Mode Off)  Tutorial 1 video  Quiz 1  Hands on experimentation  Part 2 experiment (Editing Mode On)  Tutorial 2 video  Quiz 2  Hands on experimentation  Scenario  Part 3 Interview Questions  Post-Survey Observation Methods used:  Note Taking  Audio recording  Camtasia video screen capture

Pre-Survey : Signing Sheet for Participant

Participant Log

Part 1 experiment : Editing Mode Off Tutorial 1: Quiz1:

Part 2 experiment : Editing Mode On Tutorial 2: Quiz2:

Part 2 experiment : Editing Mode On

Part 3: Open Ended Questions

Post-Survey Questionnaire

User Study Participants Locations:Library/UCI housing Total Number of participants: 8 Average Time for User Study: 36min 30sec

Heuristic Evaluation No alphabetical indexing of topics The background colors of program boxes sometimes make the label unreadable grant titles are not easily readable and go out of screen sometimes Font size is not adjustable

Heuristic Evaluation Grey Text not legibleTitles of topics overlap when dragged together

Heuristic Evaluation No maximum zoom in limit (leads to blank screen) Initial zoom abruptly increases the size of the nodes No pan function Reset/return button missing Range slider design could be better (users can miss the two slider bars)

Post-Experiment Survey Post-experiment questionnaire on System Usability Scale (SUS) Average rating of 10 questions: 3.16

Preliminary Results (1 ) Users have the need for more information associated with each nodes. They see the interface more than just simple visualization, but a way to access more information through the visualization easily. The range slider is hard to understand and operate. Many participants didn’t really understand the functionality because they used it for totally other purposes. Also, it’s hard to operate because the two bars are not visible enough to grab attention. It is cumbersome and inefficient to look through topics in circular manner. There should be easier way to navigate through the topics, such as typing to find interested topic; tagging interested topic(s) so that it’s easier to find later when you need to merge them.

Preliminary Results (2 ) Positive feedback on visualization Display of topics and nodes The fluid animation Overall visual organization Negative feedback on visualization Nodes are clumped together Font size is too small to read Constant zoom-in and zoom-out: have to zoom out to see the bigger picture, and zoom in to see the small fonts Orientations of the alignment of texts are hard to read because the user have to make big movement of their heads in order to read Some color codes are too dark to differentiate the text and color When zoom in, there’s no hand tool to move/pan. When in editing mode, rotating function disappears.

Preliminary Results (3 ) Merging function: visuals and implementation The visualization sometimes has glitches: From time to time, when two or more topics are combined, the topics overlap each other. The combined topic is usually too long to fit the screen. A suggested way to improve the visualization of combined topics: put one topic above another as a vertical list instead of putting them side by side. For the AND/OR implementation of the merging function, there are mixed views. In general, they both make sense. Hiding function: mixed views Some participants felt hiding is useful because you can get rid of irrelevant topics. Others mentioned that they don’t want to waste time to hide topics one by one.

Preliminary Results (4 ) Overall comments on usefulness of the application "It’s nice to see the graphics of the thing, but... they should have it hybrid... The way it’s displayed, it’s not good for long term use." Miscellaneous There is no reset / go back button to go to original state. Prone to error "I want to drag “security-attack” to merge- I accidentally hit “security-attack” (and the topics got hidden), so I’m going to hit “security-attack” to unhide it…" Inconsistency between functions and visualization "The ones I combined are not combined anymore.. Perhaps what I combined should stay there even though I move through the different topics and programs."

Timeline Week8 Finalize Results of the Experimental Study Week9 Find Solutions to the Problems Revealed by Results Begin to draft the final report Week10 Complete Draft of the Final Report, start Appendices Week11 Finalize Report and Submit