Evaluation of a Visualization System for Information Retrieval at the Front and the Back End Gregory B. Newby Sch of Information and Lib. Science U. of N. Carolina at Chapel Hill ASIS 1999 Midyear Meeting

Plan of the Talk Justification for attempting information visualization for IR environments The back end: information space techniques for information representation The front end: a visual, navigable interface for information retrieval: Yavi The evaluation of Yavi Current efforts & challenges

Why visualize? Many arguments have been made to visualize information, and to use visual metaphors or interfaces A more pressing question: why use a visual interface for IR? –People can use text interfaces, can’t they? –Too much CPU, too slow –No standards for developing software

Possible answers to “why visualize for IR?” All interfaces require some learning curve. Viz. might help lessen the curve. Consider how long it takes to learn a text interface The “video game” generation will have less difficulty with 2- or 3-D navigation Software and hardware are now mature enough to have much fancier visualization With comm. Standards, the interface and back end can be semi-independent

The back end: “Information space” Build term by document matrix Term relations based on tendency to cooccur, via eigensystems analysis Documents at the center of the terms they contain We can visualize any 3 dimensions Similar to VSM, but non-orthogonal Similar to LSI, but term- based, not term- document

The front end: Yavi Visualizes unstructured text (no hierarchies, controlled vocabulary, etc.) Interactive, real-time navigation Keystroke or mouse commands Standards based development (OpenGL + GLUT): will work on any modern system Gets input from the back end IR system, which can also be used for “standard” text queries

Let’s take a look… Reads in multiple files generated by the Information Space system For the user evaluation, respondents were given a training session in the functionality of the system and basis for term and document locations

The evaluation Basic question: is the interface usable? Method: field experiment Subjects: 15 LIS students, paid about $15 each (as part of their participation as TREC interactive track subjects) Process: –Train together –Perform two tasks alone in the SILS computer laboratory

The tasks, the questions Familiar task: evaluate an information space of the SILS required courses, derived from the short course descriptions Unfamiliar task: evaluate an information space concerned with TREC topic 116 Questions –Do term and document locations make sense? –Can you find a document? Rate it (topic 116) – Rate the overall system

The basis of the information spaces: Topic 116 Topic: Generic Drug Substitutions Description: Document will identify a generic drug which can be substituted for a brand name drug in the treatment of at least one medical condition. Summary: Document will identify a generic drug which can be substituted for a brand name drug in the treatment of at least one medical condition.

The basis of the information space: SILS courses 70 Organizing and Retrieving Information (3). Viles, Wildemuth. Prerequisite: INLS 50. Methods for organizing and retrieving information, including using existing databases and the construction of a database using a database management software package. 80 Data Communication (3). Dempsey, Newby. Prerequisite: INLS 50. Examines the functions of data communication networks such as the Internet for communication, accessing remote resources, and information searching and retrieval. Explores emerging multimedia applications and their potential uses.

The outcomes People could use the system, although not with great confidence They were happy with the arrangement of terms for topic 116 They were less happy with the arrangements of documents for the catalog They saw overall promise for the system, but were concerned about various aspects

What’s next for the interface?  Limiting rotation, especially about the Y axis, to lessen disorientation  Providing visual reference points (backgrounds, terrain, etc.) to help enhance depth perception  Enabling viewing of relationships beyond similarity, e.g., by selecting a document and seeing all terms in it  De-emphasizing keystroke commands in favor of on- screen heads-up menus or pull-down menus  Zooming and expanding/exploding the space was available, but underutilized. Train to use these features.

What’s next for the back end? Continued participation in TREC Completely rewritten software: more modular, able to handle multiple data types, representation schemes and retrieval methods (e.g., Boolean is a special case of VSM, VSM a special case of LSI, information space a special case of LSI) Near real-time operation for queries, ready for futher user evaluation & TREC

Conclusions Yavi, the front end, is usable Visualization v. text: still don’t know Challenges to effective use: –Understanding how the system works –Realizing 3D effects v. 2D view –How much to display? –How to display/represent items