1. Emergence and Evolution of Spatial Hypertext Frank Shipman Department of Computer Science & Center for the Study of Digital Libraries Texas A&M University

2. What is Spatial Hypertext? Hypertext but spatial … so what is hypertext? Hypertext (or hypermedia) is: 1.systems that present the same text (media) in multiple contexts 2.systems that enable the communication of relationships between documents

3. Emergence of Spatial Hypertext

4. Early Timeline of Hypertext 1945 – Vannevar Bush describes the Memex in “As We May Think” 1960s – Douglas Engelbart creates Augment 1960s – Ted Nelson uses the term “hypertext” to describe interconnected body of literature 1980s – Many hypertext systems –KMS, Document Examiner, HyperTies, Guide, NoteCards, Storyspace, Thoth, HAM, VNS, … –Growing use of the map to provide context

5. Page-Based Hypertext Frank Shipman Dr. Shipman has been researching hypertext, computer-supported cooperative work, and intelligent user interfaces since 1987 at Baylor College of Medicine, University of Colorado, Xerox PARC, and now Texas A&M University. The field of hypertext includes computer scientists, literary theorists, and writers. The first ACM Hypertext Conference took place in 1987 … Texas A&M University, located in College Station, has 43,000 students …

7. Map-Based Hypertext Observations of Xerox NoteCards activities found heavy use of maps. Aquanet designed to make the map the primary interface (instead of browser) Late 80s – map-based hypertext –gIBIS, Aquanet, Sepia –Argumentation and knowledge building –Schemas as map legends

8. Map-Based Hypertext First ACM Hypertext Conference College Station Texas A&M University Xerox PARC University of Colorado Baylor College of Medicine Frank Shipman hypertext computer-supported cooperative work intelligent user interfaces

13. Spatial Hypertext Observations of Aquanet activities showed links implied rather than explicitly expressed. VIKI designed to support building and manipulating implicit spatial relations 1993 – first spatial hypertext –VIKI: a visual and kinesthetic analysis tool Emerged into research area: –HyperMap, CAOS, Manufactur, VKB, Tinderbox, ART, …

14. Spatial Hypertext First ACM Hypertext Conference College Station Texas A&M University Xerox PARC University of Colorado Baylor College of Medicine Frank Shipman hypertext computer-supported cooperative work intelligent user interfaces Frank Shipman hypertext Texas A&M University

15. Hypertext and Expression Hypertext allows expression of interconnections between information. –References, manuals, argumentation, literature Navigational and semantic links require explicit expression in formal representation. –New ideas/designs often start as tacit –Expression hindered by barrier of initial entry Aiding creative expression –Ease of “trying out” interconnections –Constructive ambiguity

18. VIKI Cathy Marshall, Frank Shipman, Jim Coombs, and Mark LeMere Xerox PARC and Texas A&M University 1993-1997

22. Structure Recognition in VIKI Goal: to support users creation and use of implicit structure. Common structures from study of computational and paper-based spaces –Stacks –Lists –Composites Spatial parser to recognize structures that users perceive

23. Object Characteristics Used for Recognition width heighttype x, y position

24. Type Assignments for Recognized Structures StructureDescriptionType assigned StackOverlapping, homogenous Same as components ListAligned, homogenous Same as components CompositesAligned, heterogenous New type HeapOverlapping, heterogenous New type

25. Example Layout

26. Vertical List of Composite Vertical List of Horizontal List of

27. Object Position, Extent, and Type Information Resulting Spatial Parse Tree initializes tables and statistics read statistics Specialist Pipeline stack vertical list horizontal list composite vertical list horizontal list add new types Blackboard of Shared Information User-defined object types Statistics on types System-defined composite types Strategist Generates Order of Specialists

28. Using Recognized Structure VIKI uses recognized structure to provide access to implicit structure support users in formalizing structure Aid for users interacting with ephemeral structures creating composites and collections

29. Step 1Step 2Step 3Step 4

30. Example Application: Path Authoring in VIKI

31. Scaling: Non-linear Views

32. 1 st Generation Spatial Hypertext Support evolving interpretation through emergent visual languages Recognition of visual structure to aid expression Common tasks –Information collection, organization, and sharing (personal & collaborative) –Process of information triage

33. Visual Knowledge Builder Frank Shipman, Haowei Hsieh, J. Michael Moore, Anna Zacchi, Robert Airhart, Raghu Akkapeddi, Preetam Maloor, Divya Shah, Kevin Gupton Center for the Study of Digital Libraries & Department of Computer Science Texas A&M University 1997-present

34. 2 nd Generation Spatial Hypertext Support for longer-term and larger-scale tasks –Navigable history –Links between spatial hypertexts –Semi-structured metadata representation –Metadata peeling and applicators Support for presentation-oriented spatial hypertexts –Greater visual expressiveness, additional media –Spatial hypertext via http servers –Implicit types for objects

38. Navigable History Problem: inconsistent use of visual cues, interpreting ambiguous layouts Solution: return to context of interpretation Edit history with multiple navigation methods: –VCR, slider (relative), timestamp (absolute), per object/event navigation “Constructive” notion of information space

39. Links through Space and Time Why add links? –Links within information space release author from strict hierarchy –Links to other spaces aid scale / distribution Navigational issue: –Navigation in space is more complicated than navigation on page –How to backtrack? What to keep? Addition: Links can point into history of space

40. Personal Collection Creation and Use Getting content in VKB –Embedded Search for NSDL and Google –Drag-and-drop file system folders –Metadata peeling for files, jpg, mp3, search results Comprehension and modification of content –Metadata visualization of NSDL search results –Metadata extraction and applicators –Mouse-based browsing of content (including mp3 collections)

43. Metadata Extraction and Application Goal: to allow easy and consistent metadata authoring. Select objects as source for extracting metadata attributes and values Menubar of applicators is updated to allow attaching same metadata to other objects.

44. Experiences Note-taking, individual authoring tasks Organizational tasks –Project management –Conference organization Group authoring tasks –Poetry, scripts, dual-voice essays “Real”-world use –Students, researchers, writers, poets

50. Wide-Area Distributed Spatial Hypertext

51. Summary: VKB Characteristics of first generation use –Authors & readers are the same person or group –Information space is byproduct rather than artifact New directions –Presentation-oriented spatial hypertext –Wide-area distributed spatial hypertext –Uses of navigable history and history links in documents

52. VKB Finale VKB includes features to enable the creation and use of personal collections. –Embedded search, drag-and-drop folders, and metadata peeling to get information into the system –Metadata visualization, metadata extraction and applicators, and mouse-based browsing of contents for comprehension and use –Study indicates people see value in spatial hypertext for collecting and organizing information resources

53. More Spatial Hypertext Systems

54. ART used for Video Analysis Kumiyo Nakakoji & Yasuhiro Yamamoto Supporting linear authoring tasks in space

55. Hyper-Hitchcock Frank Shipman, Andreas Girgensohn, Lynn Wilcox Workspace used for authoring hypervideo Explicit links for navigational structure Spatial layout for ordering of clips into linear video sequences

56. HyperMap Antoon Verhoeven & Kai Warendorf Placing information in spaceSpatial navigation trails

57. Tinderbox Mark Bernstein

58. Topos 3D Environment Michael Bang Nielsen & Peter Ørbæk Spatial parsing in three dimensions

59. VITE: Visualization & Parsing Haowei Hsieh

60. WARP: Web-based Multi-model Adaptive Spatial Hypertext Luis Francisco-Revilla

61. Still more spatial hypertext systems CAOS (Olav Reinart, Dirk Bucka-Lassen, Claus Pedersen, Peter Nurnberg) –synchronous spatial hypertext and incremental spatial parsing ConceptLab (Rosemary Simpson) –modifiable visual languages for displaying multiple points-of-view (POV) and levels- of-detail (LOD)

62. Conclusions

63. Summary (Spatial Hypertext) Spatial hypertext enables emergent expression –Evolving visual languages –Collection, analysis and organization tasks Features of spatial hypertexts –Multitasking via collections or non-linear views –Implicit structure recognition –Navigable history, links through space and time Second generation spatial hypertext supports presentation & distribution as well –Publication-oriented spatial hypertexts

64. Resources on Spatial Hypertext Workshops on Spatial Hypertext –www.csdl.tamu.edu/~shipman/SpatialHypertextwww.csdl.tamu.edu/~shipman/SpatialHypertext VKB available at: –www.csdl.tamu.edu/VKB/www.csdl.tamu.edu/VKB/