Visualization and Spatial Hypertext Haowei Hsieh Center for the Study of Digital Libraries Texas A&M University CPSC 436, 9/28/2006

Information Life Cycle Comprehension Skimming & Reading Modification Annotation & Authoring Location Searching & Browsing Located resources must be understood to be evaluated Understanding one document may require other documents or result in further information requests Added content influences further access Reading leads to annotation, note taking, and writing

Formality of Information People naturally process informal information People naturally process informal information By perception, association, abstraction By perception, association, abstraction Computers process formalized information Computers process formalized information Database (data tables, records, fields) Database (data tables, records, fields) Metadata (attributes, values) Metadata (attributes, values) Text Note Text & Graphics Note A Node B Web Hypertext Less formalMore formal Database Superclass Subclass Object Frames & Semantic Nets

Challenges Design for humans or computers? Design for humans or computers? Humans: Humans: Good at abstraction and visual perception Good at abstraction and visual perception Handle information less formally Handle information less formally Computers: Computers: Good at repetitive computation Good at repetitive computation Data needs to be formalized Data needs to be formalized

Research Projects Visual Knowledge Builder (VKB) Visual Knowledge Builder (VKB) An information workspace providing free-form interaction and visual expression for knowledge work with unstructured information An information workspace providing free-form interaction and visual expression for knowledge work with unstructured information VITE VITE An information workspace supporting two-way mappings to work with structured information An information workspace supporting two-way mappings to work with structured information

Related Research Visual languages Visual languages Specify and present relationships Specify and present relationships Express relationships in visual workspaces Express relationships in visual workspaces Explicitly via schema based visual languages Explicitly via schema based visual languages Implicitly via patterns and spatial arrangement Implicitly via patterns and spatial arrangement

Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems NoteCardsKMSAquanetVNSTivoliHOSVIKIVITEVKB

Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems VITENoteCards Aquanet VIKI VKB

Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems NoteCardsKMSAquanetVNSTivoliHOSVIKIVITEVKB

Related Research Hypermedia/hypertext Hypermedia/hypertext Node and link hypertext Node and link hypertext Map-based hypertext Map-based hypertext Spatial hypertext Spatial hypertext Evolution of hypermedia systems Evolution of hypermedia systems NoteCardsKMSAquanetVNSTivoliHOSVIKIVITEVKB

Visual Knowledge Builder (VKB) Information objects: visual data representation Information objects: visual data representation Collections -- hierarchy of 2-d spaces Collections -- hierarchy of 2-d spaces Visual attributes for user interpretation Visual attributes for user interpretation Import information by drag & drop, copy/paste Import information by drag & drop, copy/paste Short Demo Short Demo

VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history

VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history Lab Administrators username: haowei group: power user

VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history

VKB Advanced Features Spatial parser (structure recognition) Spatial parser (structure recognition) User-defined attributes User-defined attributes Integrated search (NSDL and Google) Integrated search (NSDL and Google) Navigable history Navigable history

Example Applications Organization Organization

Example Applications Information gathering and collection Information gathering and collection

Example Applications Project management Project management

VKB Summary Support the information life cycle Support the information life cycle Embedded search and easy import Embedded search and easy import Visual manipulation and expression Visual manipulation and expression Support for incremental formalization Support for incremental formalization Gradually transform informal information into semi-formal or formal information Gradually transform informal information into semi-formal or formal information Software freely available at Software freely available athttp://

VITE An information workspace for manipulating structured information in a semi-formal setting An information workspace for manipulating structured information in a semi-formal setting

Motivation Difficulties interacting with structured information Difficulties interacting with structured information Incomplete representation: deconstructive nature of formalization Incomplete representation: deconstructive nature of formalization Intermediate representation: transient states during problem solving Intermediate representation: transient states during problem solving

Two-way Mapping Mapping Data TableVisual Display Visualization Visual Parsing Two-way Mapping Link semantic content of the database with visual attributes of information objects

Semantic Attributes Visual Attributes Position, Size, Color (background and border), Text label, Border width Position, Size, Color (background and border), Text label, Border width Semantic Attributes Title, Level, Start Time, Instructor, Classroom, etc. Title, Level, Start Time, Instructor, Classroom, etc. Visual Attributes

Semantic Value Visual Value Mapping assignments Mapping assignments Day of Week: X Position Day of Week: X Position Start: Y Position Start: Y Position Difficulty: BG Color Difficulty: BG Color Title: Text Label Title: Text Label Start: Text Label Start: Text Label End: Text Label End: Text Label Value assignment e.g. Difficulty Value assignment e.g. Difficulty Level 1:Green Level 1:Green Level 2:Yellow Level 2:Yellow Level 3:Red Level 3:Red Mathematics Start: 1000 End: 1100

VITE Architecture Visual Workspace Mapping Engine Data Storage Mapping Designer Miscellaneous Interface Mapping Assignment Interface Mapping Assistant Mapping Profile User Inputs Visual Display Visual Parser Visual Renderer

VITE Interface Workspace Workspace Mapping Designer Mapping Designer Mapping Assignment Mapping Assignment Continuous/Discrete settings Continuous/Discrete settings Miscellaneous Interfaces Miscellaneous Interfaces

Mapping Designer Visual Attributes Selected Semantic Attribute Available Semantic Attribute

Mapping Assignment 1. Continuous value 2. Continuous color 3. Discrete value 4. Discrete color

Mapping Assignment 1/4 Estimated Range for Selected Visual Attribute Selected Visual Attribute : Y Size Select Attribute : MarketValue Available Value Range from Scanning the Database Select Mapping Style: Continuous Mapping Preview

Mapping Assignment 2/4 Default Color Range for Selected Visual Attribute Selected Visual Attribute : Background Color Select Attribute : MarketValue Available Value Range from Scanning the Database Select Mapping Style: Continuous Mapping Preview

Mapping Assignment 3/4 Default Values for Selected Visual Attribute Selected Visual Attribute : X Position Select Attribute : POS Available Value from Scanning the Database Select Mapping Style: Discrete

Mapping Assignment 4/4 Default Values for Selected Visual Attribute Selected Visual Attribute : Background Color Select Attribute : POS Available Values from Scanning the Database Select Mapping Style: Discrete

VITE Demo Class scheduling example used in the tutorial Class scheduling example used in the tutorial

User Study Investigate Investigate How people solve problems using visual workspaces How people solve problems using visual workspaces How people use two-way mappings How people use two-way mappings Observe Observe General use and selection of visual attributes General use and selection of visual attributes Identify Identify Problems when dealing with two-way mapping Problems when dealing with two-way mapping Desired enhancements through feedback Desired enhancements through feedback

Tasks Task 1: Flight approach control Task 1: Flight approach control User acts as an air traffic controller User acts as an air traffic controller schedule airplanes to avoid conflict schedule airplanes to avoid conflict Task 2: Fantasy basketball Task 2: Fantasy basketball User acts as a team manager of a fantasy basketball team User acts as a team manager of a fantasy basketball team Select 5 players within a limited budget Select 5 players within a limited budget

Task 1: Data Attributes Arrival: Arrival time Runway: Pre-assigned runway Flight: Flight Number Category:Airliner, Regional passenger plane, or Cargo plane Airline: Airline who operates this flight Airplane: Type of the aircraft (Make/Model) AP_Speed: Approach speed RW_Length: Required runway length. Capacity: Maximum number of passenger Passengers: Number of passengers on board Departure: Departure time Engine_Type: Type of engine of the aircraft, i.e., Jet, Turbo, etc. No_Engines: Number of engines Gate: Preplanned gate assignment Status: Flight status

Task 1: Example Mapping

Task 1: Example Space

Task 2: Data Attributes Market Value : Current salary to recruit the player POS : Position the player plays Decision : Decision to choose, reject, or still consider the player. Athlete: The name of the player Team : Where the team comes from 3PT : 3 point shots made REB : Rebounds Min : Minimum field goal AST : Assists BLK : Blocked Shots STL : Steals TO :Turnovers FTM-A : Free Throws Made-Attempted FT% : Free Throw Percentage FGM-A : Field Goals Made-Attempted FG% : Field Goal Percentage Prev : Salary of the player's previous contract Change : Change of the player's salary between current market value and previous salary.

Task 2: Example Space

Task 2: Visual Problem Solving

Task 2: Creating Reference

Task 2: Visual Sorting Object height is mapped to a critical attribute. The user need to find a proper object by comparing their height. Object height is mapped to a critical attribute. The user need to find a proper object by comparing their height. Step 1: select objects to be sorted Step 1: select objects to be sorted

Task 2: Visual Sorting Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Step 2: make the selection into a pile by Stack function Step 2: make the selection into a pile by Stack function

Task 2: Visual Sorting Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Object height is mapped to a critical value. The user need to find a proper object by comparing their height. Step 3: Align the selection at top with Align Y - Top function. Step 3: Align the selection at top with Align Y - Top function. A list of objects handy for comparing their height and sorting A list of objects handy for comparing their height and sorting

Task 2: Comparing

Task 2: Result

Task 2: Similar Strategy

Observations Users’ preference of visual properties Users’ preference of visual properties Position > (Size, Color) Position > (Size, Color) Text label is inevitable Text label is inevitable Use and selection of visual properties: Use and selection of visual properties: Unmapped visual properties: temporary memory Unmapped visual properties: temporary memory Text Label: for textural description and feedback Text Label: for textural description and feedback Position: to hold status, sort, distribute, or categorize Position: to hold status, sort, distribute, or categorize Color: to show status, categorization, or reveal value distribution Color: to show status, categorization, or reveal value distribution Size: to compare value (numeric) or show categorization (discrete) Size: to compare value (numeric) or show categorization (discrete) Border Width: rarely used, because too few values available, and it effect the usefulness of border color Border Width: rarely used, because too few values available, and it effect the usefulness of border color

VITE Summary Two-way mapping enables semi-formal manipulation of formalized information Two-way mapping enables semi-formal manipulation of formalized information VITE supports users in developing advanced problem solving strategies VITE supports users in developing advanced problem solving strategies VITE provides users new ways of interacting with structured information VITE provides users new ways of interacting with structured information Users identified additional features that would support this approach Users identified additional features that would support this approach

Conclusion Information workspaces support all phases of the information life cycle Information workspaces support all phases of the information life cycle Incremental formalization facilitate the transformation of informal information into more formal knowledge representations (VKB) Incremental formalization facilitate the transformation of informal information into more formal knowledge representations (VKB) Two-way mappings enable interaction with formalized information in semi-formal environments (VITE) Two-way mappings enable interaction with formalized information in semi-formal environments (VITE)