Emergence and Evolution of Spatial Hypertext Frank Shipman Department of Computer Science & Center for the Study of Digital Libraries Texas A&M University

What is Spatial Hypertext? Hypertext but spatial … so what is hypertext? Hypertext (or hypermedia) is: 1.systems that present the same text (media) in multiple contexts 2.systems that enable the communication of relationships between documents

Emergence of Spatial Hypertext

Early Timeline of Hypertext 1945 – Vannevar Bush describes the Memex in “As We May Think” 1960s – Douglas Engelbart creates Augment 1960s – Ted Nelson uses the term “hypertext” to describe interconnected body of literature 1980s – Many hypertext systems –KMS, Document Examiner, HyperTies, Guide, NoteCards, Storyspace, Thoth, HAM, VNS, … –Growing use of the map to provide context

Page-Based Hypertext Frank Shipman Dr. Shipman has been researching hypertext, computer-supported cooperative work, and intelligent user interfaces since 1987 at Baylor College of Medicine, University of Colorado, Xerox PARC, and now Texas A&M University. The field of hypertext includes computer scientists, literary theorists, and writers. The first ACM Hypertext Conference took place in 1987 … Texas A&M University, located in College Station, has 43,000 students …

Map-Based Hypertext Observations of Xerox NoteCards activities found heavy use of maps. Aquanet designed to make the map the primary interface (instead of browser) Late 80s – map-based hypertext –gIBIS, Aquanet, Sepia –Argumentation and knowledge building –Schemas as map legends

Map-Based Hypertext First ACM Hypertext Conference College Station Texas A&M University Xerox PARC University of Colorado Baylor College of Medicine Frank Shipman hypertext computer-supported cooperative work intelligent user interfaces

Spatial Hypertext Observations of Aquanet activities showed links implied rather than explicitly expressed. VIKI designed to support building and manipulating implicit spatial relations 1993 – first spatial hypertext –VIKI: a visual and kinesthetic analysis tool Emerged into research area: –HyperMap, CAOS, Manufactur, VKB, Tinderbox, ART, …

Spatial Hypertext First ACM Hypertext Conference College Station Texas A&M University Xerox PARC University of Colorado Baylor College of Medicine Frank Shipman hypertext computer-supported cooperative work intelligent user interfaces Frank Shipman hypertext Texas A&M University

Hypertext and Expression Hypertext allows expression of interconnections between information. –References, manuals, argumentation, literature Navigational and semantic links require explicit expression in formal representation. –New ideas/designs often start as tacit –Expression hindered by barrier of initial entry Aiding creative expression –Ease of “trying out” interconnections –Constructive ambiguity

VIKI Cathy Marshall, Frank Shipman, Jim Coombs, and Mark LeMere Xerox PARC and Texas A&M University

Structure Recognition in VIKI Goal: to support users creation and use of implicit structure. Common structures from study of computational and paper-based spaces –Stacks –Lists –Composites Spatial parser to recognize structures that users perceive

Object Characteristics Used for Recognition width heighttype x, y position

Type Assignments for Recognized Structures StructureDescriptionType assigned StackOverlapping, homogenous Same as components ListAligned, homogenous Same as components CompositesAligned, heterogenous New type HeapOverlapping, heterogenous New type

Example Layout

Vertical List of Composite Vertical List of Horizontal List of

Object Position, Extent, and Type Information Resulting Spatial Parse Tree initializes tables and statistics read statistics Specialist Pipeline stack vertical list horizontal list composite vertical list horizontal list add new types Blackboard of Shared Information User-defined object types Statistics on types System-defined composite types Strategist Generates Order of Specialists

Using Recognized Structure VIKI uses recognized structure to provide access to implicit structure support users in formalizing structure Aid for users interacting with ephemeral structures creating composites and collections

Step 1Step 2Step 3Step 4

Example Application: Path Authoring in VIKI

Scaling: Non-linear Views

1 st Generation Spatial Hypertext Support evolving interpretation through emergent visual languages Recognition of visual structure to aid expression Common tasks –Information collection, organization, and sharing (personal & collaborative) –Process of information triage

Visual Knowledge Builder Frank Shipman, Haowei Hsieh, J. Michael Moore, Anna Zacchi, Robert Airhart, Raghu Akkapeddi, Preetam Maloor, Divya Shah, Kevin Gupton Center for the Study of Digital Libraries & Department of Computer Science Texas A&M University 1997-present

2 nd Generation Spatial Hypertext Support for longer-term and larger-scale tasks –Navigable history –Links between spatial hypertexts –Semi-structured metadata representation –Metadata peeling and applicators Support for presentation-oriented spatial hypertexts –Greater visual expressiveness, additional media –Spatial hypertext via http servers –Implicit types for objects

Navigable History Problem: inconsistent use of visual cues, interpreting ambiguous layouts Solution: return to context of interpretation Edit history with multiple navigation methods: –VCR, slider (relative), timestamp (absolute), per object/event navigation “Constructive” notion of information space

Links through Space and Time Why add links? –Links within information space release author from strict hierarchy –Links to other spaces aid scale / distribution Navigational issue: –Navigation in space is more complicated than navigation on page –How to backtrack? What to keep? Addition: Links can point into history of space

Personal Collection Creation and Use Getting content in VKB –Embedded Search for NSDL and Google –Drag-and-drop file system folders –Metadata peeling for files, jpg, mp3, search results Comprehension and modification of content –Metadata visualization of NSDL search results –Metadata extraction and applicators –Mouse-based browsing of content (including mp3 collections)

Metadata Extraction and Application Goal: to allow easy and consistent metadata authoring. Select objects as source for extracting metadata attributes and values Menubar of applicators is updated to allow attaching same metadata to other objects.

Experiences Note-taking, individual authoring tasks Organizational tasks –Project management –Conference organization Group authoring tasks –Poetry, scripts, dual-voice essays “Real”-world use –Students, researchers, writers, poets

Wide-Area Distributed Spatial Hypertext

Summary: VKB Characteristics of first generation use –Authors & readers are the same person or group –Information space is byproduct rather than artifact New directions –Presentation-oriented spatial hypertext –Wide-area distributed spatial hypertext –Uses of navigable history and history links in documents

VKB Finale VKB includes features to enable the creation and use of personal collections. –Embedded search, drag-and-drop folders, and metadata peeling to get information into the system –Metadata visualization, metadata extraction and applicators, and mouse-based browsing of contents for comprehension and use –Study indicates people see value in spatial hypertext for collecting and organizing information resources

More Spatial Hypertext Systems

ART used for Video Analysis Kumiyo Nakakoji & Yasuhiro Yamamoto Supporting linear authoring tasks in space

Hyper-Hitchcock Frank Shipman, Andreas Girgensohn, Lynn Wilcox Workspace used for authoring hypervideo Explicit links for navigational structure Spatial layout for ordering of clips into linear video sequences

HyperMap Antoon Verhoeven & Kai Warendorf Placing information in spaceSpatial navigation trails

Tinderbox Mark Bernstein

Topos 3D Environment Michael Bang Nielsen & Peter Ørbæk Spatial parsing in three dimensions

VITE: Visualization & Parsing Haowei Hsieh

WARP: Web-based Multi-model Adaptive Spatial Hypertext Luis Francisco-Revilla

Still more spatial hypertext systems CAOS (Olav Reinart, Dirk Bucka-Lassen, Claus Pedersen, Peter Nurnberg) –synchronous spatial hypertext and incremental spatial parsing ConceptLab (Rosemary Simpson) –modifiable visual languages for displaying multiple points-of-view (POV) and levels- of-detail (LOD)

Conclusions

Summary (Spatial Hypertext) Spatial hypertext enables emergent expression –Evolving visual languages –Collection, analysis and organization tasks Features of spatial hypertexts –Multitasking via collections or non-linear views –Implicit structure recognition –Navigable history, links through space and time Second generation spatial hypertext supports presentation & distribution as well –Publication-oriented spatial hypertexts

Resources on Spatial Hypertext Workshops on Spatial Hypertext – VKB available at: –