1. ETANA-DL ( Electronic Tools and Near Eastern Archives Digital Library) Edward A. Fox, Virginia Tech James W. Flanagan, Case Western Reserve U. AIA 106 th Annual Meeting, Boston, Jan. 2005 fox@vt.edu http://fox.cs.vt.edu http://fox.cs.vt.edu/talks/

2. Outline Acknowledgements ETANA-DL High Level Overview Harvesting, Open Archives Initiative OCKHAM, Reference Models OAIS (ISO Standard for Archiving) 5S (Digital Library Framework) ETANA-DL Approach, Services, Integration Conclusions

3. NSF ITR Funding IT Research Digital library: Integration of DB, HCI, HT, IR, LIS, MM, … Complexity! Variety! Distributed! => 5S Framework + OAI / ODL Archaeology Research Multiple sites Multiple kinds of artifacts Multiple terminologies General/special services Multiple views Hypothesis testing Rapid publication

4. Map courtesy: www.enchantedlearning.com Initial ETANA-DL Member Locations Virginia Tech Mississippi State University Vanderbilt University Canadian University College Walla Walla College Andrews University CWRU Willamette University

5. Acknowledgements Contributors: Karen Borstad, MPP Douglas Clark, Walla Walla College Larry Herr, Canadian University College Christopher Holland, LRP Paul Jacobs, Mississippi State U. Stan LaBianca, Andrews U. David McCreery, Willamette U. David Schloen, U. of Chicago Randall Younker, Andrews U.... Team: Joanne Eustis, CWRU Weiguo Fan, Virginia Tech Nick Fischio, CWRU Paul Gherman, Vanderbilt U. Marcos Goncalves, Virginia Tech Doug Gorton, Virginia Tech (CS4624) Douglas Knight, Vanderbilt U. Likhita Krishnamurthy, VT (CS5604) Ming Luo, Virginia Tech Ananth Raghavan, VT (CS5604) Divya Rangarajan, VT (Ind. Study) Unni Ravindranathan, Virginia Tech Jack Sasson, Vanderbilt U. Rao Shen, Virginia Tech Ricardo Torres, U. Campinas, Brazil Srinivas Vemuri, Virginia Tech

9. Lahav Website

10. Megiddo Opening Screen

11. Locus Screen: Pictures View all

12. Area Screen

14. ETANA-DL Website

15. ETANA-DL Architecture UsersServicesData ETANA-DL Union ServicesUsers DigBaseDigKit

16. ETANA-DL Architecture DigBase and DigKit Lahav Nimrin Umayri Hisban Megiddo Jalul New Sites DATABASEWRAPPERSDATABASEWRAPPERS ETANA-DL UNION CATALOG Search USERINTERFACEUSERINTERFACE Browse Recommend Note Personalize Review Visualizations Archaeology Specific 1 st Prototype …

17. Open Archives Initiative OAI www.openarchives.org

18. Open Archives Initiative (OAI) Protocol for Metadata Harvesting Black Box Perspective OA 1OA 2OA 4OA 3OA 5OA 6OA 7

19. OAI = Technical Umbrella for Practical Interoperability… Reference Libraries Publishers E-Print Archives …that can be exploited by different communities Museums

20. OAI Repository Perspective Required: Protocol DO MDO

21. Discovery Current Awareness Preservation Service Providers Data Providers Metadata harvesting OAI: Data & Service Providers

22. Data and Service Providers Data Providers  possess metadata and share it (internally / externally)  via well-defined OAI protocols (e.g., database servers) Service Providers  harvest and preserve data from Data Providers  provide higher-level services to users  (e.g., search engines) Who will fit where in ETANA-DL?  Data Provider – YOUR PROJECT  Service Provider – ETANA-DL

24. OCKHAM Simplicity (a la OCCAM’s razor) Support by Mellon and DLF Four main ideas: 1. Components 2. Lightweight protocols 3. Open reference models (e.g., 5S, OAIS) 4. Community perspective and involvement Funded by NSF in NSDL, with P2P

25. Reference Models Reference Model: a common vocabulary and description of components, services, and inter-relationships that comprise a system under consideration Useful as a tool to foster consensus and common understanding in a time of rapid change and/or disagreement

26. OAIS RLG Pages - www.rlg.org

27. OAIS RLG Examples – 1 of 2

28. OAIS RLG Examples – 2 of 2

29. http://ssdoo.gsfc. nasa.gov/nost/iso as/

34. Informal 5S Definitions DLs are complex systems that help satisfy info needs of users (societies) provide info services (scenarios) organize info in usable ways (structures) present info in usable ways (spaces) communicate info with users (streams)

35. 5S SsExamplesObjectives Streams Text; video; audio; image Describes properties of the DL content such as encoding and language for textual material or particular forms of multimedia data Structures Collection; catalog; hypertext; document; metadata Specifies organizational aspects of the DL content Spaces Measure; measurable, topological, vector, probabilistic Defines logical and presentational views of several DL components Scenarios Searching, browsing, recommending Details the behavior of DL services Societies Service managers, learners, teachers, etc. Defines managers, responsible for running DL services; actors, that use those services; and relationships among them

36. SitePartitionSub-partitionLocusContainer LahavField I Area A8 Locus A8074 Basket 224 NimrinQuadrant NW Quadrant Value N25/W50 Locus 96 Bag 240 UmayriField A Square 7J59 Locus 001 Pail 12 5S Structural Model Organization

37. 5S Meta Model 5SGraph DL Expert DL Designer 5SL DL Model 5SLGen Practitioner Researcher Tailored DL Services Teacher c omponent pool ODLSearch, ODLBrowse, ODLRate, ODLReview, ……. Requirements (1) Analysis (2) Implementation (4) Design (3) 5SGraph5SGen Mapping Tool 5S Suite

38. ETANA-DL Architecture Union Catalog Inverted Files Services DB Index Browse Component Search Component Browse DB Other ETANA-DL Services Web Interface XOAI DigBase DB Data Mapping Component OAI Data Provider OAI Archaeological Site ETANA-DL DigKit Configure

39. Digital Object Repository Collection Minimal DL Metadata Catalog Descriptive Metadata Specification A Minimal DL in the 5S Framework Structural Metadata Specification StreamsStructuresSpacesScenariosSocieties indexing browsing searching services hypertext Structured Stream

40. 5S Archaeological DL Modeling Modeling archaeological information systems using the 5S theory to better understand the domain and design the system and the supporting services

41. StreamsStructuresSpacesScenariosSocieties indexing browsing searching services hypertext Structured Stream Descriptive Metadata specification SpaTemOrg StraDia Arch Descriptive Metadata specification ArchDO ArchObj ArchColl Arch Metadata catalog ArchDColl ArchDR Minimal ArchDL A Minimal ArchDL in the 5S Framework

42. Modeling ETANA-DL – An Archaeological DL Meta-model Text Video Audio *Site *Sub-partition *Container*Artifact*LocusRegion Taxonomies Temporal Artifact-specific Space model Structure model Metadata DrawingPhoto3D Stream model *Partition Society model Archaeologist General public Geographic space Service Manager Information Satisfaction Value added Repository building Scenario model Services Domain specific User interfaceMetric space Spatial

43. Modeling ETANA-DL – ETANA Model *Field*Pail *Bone *LocusJordan Taxonomies Space model Structure model Field record, locus sheet Figurine image (photo) Stream model Umayri Society model Archaeologist Generic public Site-specific coordinate system Web interface Vector space ETANA-DL Service Manager Searching, Browsing Annotation, binding Harvesting, Converting Scenario model Services Object comparison, marking item for analysis Archaeological periods Bone type Seed species *Square *Figurine *Quadrant*Bag *Locus Jordan Valley Nimrin *Square *Field*Basket*LocusSouthern IsraelHalif*Area *Seed Site/field plan (drawing) Preliminary/Final Report (application/pdf) Spatial

44. Overall objective of 5SGraph: Help users model their own instances of a digital library (DL) in the 5S language (5SL). A simple modeling process which enables rapid generation of digital libraries is needed.  Support non-expert users.  Speed-up development process.  Increase the quality of final product. 5SGraph: A DL Modeling Tool

45. Overview of 5SGraph Workspace (instance model) Structured toolbox (metamodel)

46. Space Model

47. Society Model

48. ETANA-DL Approach Applying and extending Digital Library (DL) techniques to solve key problems: making primary data available, data preservation, and interoperability Modeling archaeological information systems using 5S to better understand the domain and design the system and the supporting services Rapidly prototyping DLs that handle heterogeneous archaeological data using componentized frameworks:  eliciting requirements  refining metamodel and union schema  modeling sites  mapping  harvesting  providing useful services

49. Marking – writing notes for a specific user Marking Items

50. Marked Items Display Sender, Date, Object OAI ID Sender Comments Options: View Record, Add record to Items Of Interest, Re-mark item (Redirect), Unmark item (Remove item from list)

51. Discussions Page Discussions about an object View/Post messages, create new threads

52. Recommendations Items recommended on the basis of similar interests

53. ETANA-DL Searching Service Search

54. ETANA-DL Multi-dimensional Browsing 3 new sites 2 new types of artifacts

55. ETANA-DL Visual Browsing Service Visual Browse By site

56. Visual Browsing Nimrin: Topographical Drawings Full siteNorth west quadrant Square: N40/W20

57. Visual Browsing Nimrin : Square information Square: N40/W20 Locus: 86 Loci layout

58. Visual Browsing Nimrin : locus sheet

59. Visual Browsing Bab edh-Dhra' Cemetery Pottery # 25

60. Visual Browsing Bab edh-Dhra' Cemetery Pottery # 25

61. Repository1 DL1 Repository2 Union Catalog Union Repository Catalog1Catalog2 Searching Union DLDL2 archaeologists Society General Public Society Archaeologists General Public Union Society Service Browsing Service Union Service Harvesting, Mapping, Searching, Browsing, Clustering, Visualization Architecture of a Union DL

62. Union Catalog VN Catalog Union Catalog Integration Virtual Nimrin (VN) Halif DigMaster (HD) HD Catalog VN Metadata Format Mapping Tool Mapping Tool Global Metadata Format Wrapper HD Metadata Format

63. SiteArtifact TypeOriginal data source Number of records harvested Bab edh-Dhra’Potterycp6 database file786 LahavFigurineTab-delimited text file563 MadabaLocus field recordTables in Access DB786 MozanPublicationPDF files19 Nimrin Bone field recordTable in Oracle DB7419 Seed field recordTable in Oracle DB429 Locus field recordTable in Oracle DB2101 UmayriBone field record2 tables in Access DB2122 Total18404 Heterogeneous data handling

64. ETANA-DL Schema Design Bone Seed Figurine ETANA-DL Object Count Animal …… Species Name …… Description Dimensions …… Owner Subpartition Partition Locus ID Container Collection ……

65. Visualizing Components Mapper1 Composite Mapper Mapper2Mapper3Mapper4 Visual Mapping Tool Architecture

66. Data Mapping (state-of-the-art)

67. local schemaglobal schema

68. Mapping recommendation

69. Mapping confirmationMapping history

70. No recommendation for “Tomb_Area”

71. User-decided mapping

72. 5SGraph 5S Archaeology MetaModel ArchDL Expert ArchDL Designer VN Metadata Format ETANA-DL Metadata Format Mapping Tool Wrapper4VNWrapper4HD HD Metadata Format Inverted Files Services DB Index Browse Service Search Service Browse DB Other ETANA-DL Services Web Interface XOAI VN Catalog VN Catalog Union Catalog Structure Sub-model Scenario Sub-model Harvesting description Mapping description Browsing description … 5SGen Component Pool Browsing …

73. Conclusions Working on 5S book … See http://feathers.dlib.vt.edu/http://feathers.dlib.vt.edu/ Thanks to NSF ITR IIS-0325579! Please fill in and return/send survey!