NetLens: Iterative Exploration of Content-Actor Network Data Hyunmo Kang, Catherine Plaisant, Ben Bederson

Challenges of Network Data Visualization by Frank van Ham TouchGraph

Challenges of Network Data Visualization Complex Analytic Tasks Incremental data exploration Iterative query refinement Scalability Common simple UI components e.g. histogram and lists Generality Apply to any dataset matching Content-Actor model e.g. digital library, photo collections, collections, case law, etc.

Data Analysis (Content-Actor Model) Entity E1 (Content) Entity E2 (Actor) Intra-relationship Inter-relationship

Data Analysis (Content-Actor Model) Content (Paper) Actor (Author) Intra-relationship Inter-relationship

Data Analysis (Content-Actor Model) Content ( ) Actor (People) Intra-relationship Inter-relationship

Data Analysis (Content-Actor Model) Actor (Photo) Content (People or Category) Intra-relationship Inter-relationship

DEMO Screenshots at Video at

Task Analysis Search By Search Result Entity1 (Paper)Entity2 (People) Entity1’s AttributesEntity1(Paper)Entity2’s AttributesEntity2 (People) Entity1 (Paper) Search for papers by paper attributes such as year, keywords, title, conference, topics, etc. Search for papers that “cite” or “are cited by” the selected papers along with frequency Search for papers by people attributes such as author’s affiliation, institution, nationality, etc. Search for papers written by the selected authors (either conjunctive or disjunctive) Entity2 (People) Search for authors by paper attributes such as year, keywords, title, conference, topics, etc. Search for authors of the selected papers with frequency (the number of papers per each author) Search for authors by people attributes such as author’s affiliation, institution, nationality, etc. Search for academic advisors of the selected authors (either conjunctive or disjunctive)

Task Analysis Single step tasks How many papers on “User Study” were published in 1998? Who are the authors of the papers on “Virtual Reality”, which were published at the CHI 99 conference? Which paper is the most frequently cited by the papers published at the CHI 04 conference? Which author is most frequently cited in the “InfoVis” topic? How many papers were published by UMD HCIL people? Who are the authors whose nationality is Korea?

Task Analysis Multiple step tasks Evaluate individuals: - how many papers were self-referenced? - how frequently was each paper referenced by other papers? Identify communities: - what are the major paper topics published by UMD HCIL? and who in this group has the most papers in that topic? - how do UMD HCIL’s research interests change over time? and who in this group made that change? Find experts (to review papers or come to workshop): - who wrote the most papers in the InfoVis topic? and how many papers cited his papers? - whose paper in the InfoVis area is most frequently referenced by other papers? Learning about a new topic (to find a good PhD topic): - which topic has growing publications? and who contributed most to this topic last 3 years? - what are the other topics the authors in InfoVis area also get interested in? Where should I go on a sabbatical? - which country (or research group)’s authors most frequently reference my papers?

Design Challenges History and Integrated Help Sequence of interactions to accomplish a task (lost in exploration) “How did I get here?” “What does the current filtered dataset mean?”

Design Challenges Multi-layered Interface Users do not need all the windows Complexity of data and tasks Computation efficiency Users’ usage levels and their preferences Etc.

Design Challenges Data Export Integration of graph visualizing tool TreePlus Exporting methods Windows clipboard Internal graph class object Xml documents

NetLens Extension s on the left People on the right Overviews provided for all attributes (here for emotional tone on s side) Filtered to show only s related to CA energy crisis; and the people who sent them are shown on the right side. (Joint Institute for Knowledge Discovery) -

NetLens Extension (Joint Institute for Knowledge Discovery) Generality and Scalability JIKD data schema

NetLens Data Schema

NetLens System Architecture NetLens Written in C#, Piccolo toolkit Database Server MySQL ADO.NET driver for MySQL MySQL connector/NET Web Server Mac OS X Server Web API CGI, JSP (e.g. search, people’s bio, etc.)

Evaluation Heuristic Evaluation by NIST Possible directions: Usability Measure usability Speed, performance, Learnability Error rates Power Comparing range and complexity of possible queries SQL queries? Generality How easy it is to apply new datasets to NetLens

PhotoMesa Browse, Annotate, and Search Digital Images Hyunmo Kang and Ben Bederson

PhotoMesa Image Browsing Zoomable User Interface Zooming into a group of photos Zooming into a single photo PhotoMesa shows all photos in a single view Bigger preview by moving over a thumbnail Browse photos by zooming in or out Dynamic sorting and grouping

PhotoMesa Image Browsing Zoomable User Interface PhotoMesa lets you control visible photos All photos Unhidden photos Representative photos Favorite photos only Show only the representative photos for each group

PhotoMesa Image Browsing Zoomable User Interface Browse photos in “Scroll” mode with detail photo view Photo Information with EXIF Scrollable Thumbnails Panel Detail Photo Panel

PhotoMesa Annotating Add a caption and mark photo as favorite or hidden Label who is in the photo Label objects in the photo (e.g. animals, locations, etc.) People Annotation Mechanisms: Checkbox Annotation, Drag-and-drop Annotation, Hotkey Annotation Category Annotation: Create user-defined hierarchical structure of object types to annotate your photos with

PhotoMesa Annotating Bulk Annotation: Annotate multiple photos simultaneously with the same annotation mechanisms

PhotoMesa Searching By keyword By folders By people By category By year By month

PhotoMesa Photo Sharing Upload Photos Metadata e.g.) people, category, photo info, etc. Remove Update Search Web Services Browse with web browser Add comments

PhotoMesa Data Schema

PhotoMesa SQL Query Free Text Search (Find photos containing word “kang”) SELECT Photos.* FROM (Photos INNER JOIN (Categories INNER JOIN PhotosCategories ON Categories.categoryname = PhotosCategories.categoryname) ON Photos.url = PhotosCategories.url) INNER JOIN (People INNER JOIN PhotosPeople ON People.personname = PhotosPeople.personname) ON Photos.url = PhotosPeople.url WHERE (((PhotosPeople.personname) Like "*kang*") OR ((Photos.url) Like "*kang*") OR ((Photos.created) Like "*kang*") OR ((Photos.uploaded) Like "*kang*") OR ((Photos.description) Like "*kang*")) OR (((PhotosCategories.categoryname) Like "*kang*"));

PhotoMesa SQL Query Add Photo INSERT INTO Photos (url, created, uploaded, description, photomark, thumbnail, width, height) Values(“url”, “ ", “ ", “hyunmo’s trip to Seoul”, 1280, 1024); Add People INSERT INTO PhotosPeople (url, personname, x, y, time) Values(“url”, “hyunmo kang“, “0.1234”, “0.789” “ "); INSERT INTO People (personname, lastname, firstname) Values(“hyunmo kang”, “kang”, “hyunmo”);

PhotoMesa System Architecture PhotoMesa Client Written in C#, Piccolo toolkit Database Server MySQL ADO.NET driver for MySQL MySQL connector/NET Web Server Apache Web API PHP (photo upload, web services)

Questions? More visualization projects are available at PhotoMesa NetLens