Digital Library Issues and Trends William H. Mischo w-mischo@uiuc.edu Grainger Engineering Library Information Center University of Illinois at Urbana-Champaign Taken from: Technology, Scalability, and Metadata (or Not) NSDL Panel, October 13, 2003

Digital Libraries ‘Digital’, ‘Virtual’, ‘Electronic’ Library as network-based library without regard to place and time. Tendency to apply term to digital collections and resources. Digital Collections vs. Digital Libraries. Emphasis on the integration of collections and services via standards and Web protocols and practices. Distributed, heterogeneous repositories require federation and linking.

Scholarly Communication Overview E-Resources are Web-based and publisher-centric. Growth of Heterogeneous Distributed Repositories. ARL libraries spent $132 million on e-resources in 2001. 30K e-journals (TDNet), >33% available via multiple aggregators and/or publishers. Publisher’s provide value-added services and ‘branding’ of journals. Prestige of journals and publishers in Academia. Cooperation on reference linking standards (DOI, OpenURL, CrossRef). Alternative scholarly communication models - Academia institutional repositories, preprint servers.

Distributed Information Environment We live in a world of multiple, heterogeneous information repositories, resources, portals, and IR systems. OPACs: local, regional, national shared bibliographic databases. Locally mounted and remote A & I Services. Discrete publisher and vendor full-text repositories. Open OAI services (OAIster at Michigan) and preprint servers. Web search engines. Vertical and custom publisher and vendor portals (ARL Portal, DOE Information Bridge, EI Village, NSDL). Surface Web and Hidden Web. Local digital object stores, GIS systems, finding aids. Institutional Repositories (D-Space). Instructional (course) management systems (WebCT, Blackboard). David Seaman: ‘we don’t shelve by publisher, why do we expect users to search by publisher.’

Distributed Repository Issues Metasearch (broadcast and federated) over distributed, heterogeneous repositories. Publishers and vendors efforts to become one-stop-shopping portals: BigChalk, Elsevier, Ingenta, EbscoHost, AIP. Ironic need to ‘unbundle’ DLs for optimum search (IEEE IEL, ACM, EI Village, Elsevier, Wiley, Springer, Kluwer, AIP, etc.) Role of A & I Services. Tenuous? But important in Local Link Resolving. Link Resolver Servers (vendor: SFX, Encompass) via DOI, OpenURL (metadata), CrossRef. Integration of collections with reference, instructional and navigation services -TOC, collaborative mechanisms, remote reference assistance. Search and discovery navigation: Best-Match and Quorum Searching.

Digital Library Tools We have at our disposal the tools to create integrated digital libraries from the distributed resources environment: Standard retrieval environment (Web) and interface/client (Web Browser); Standard transport mechanisms to connect heterogeneous content (HTTP, OAI, SOAP); Standard metalanguages and tools for describing and transforming content and metadata (XML, DTDs & Schemas, XSLT, DC/DCQ, RDF, METS, MPEG-21); Standardized search/retrieval mechanisms (HTTP Post/Get, SQL, Z39.50, Xquery); Standard linking tools and infrastructure (DOI, OpenURL, CrossRef). Candidate set of ‘best practices’ for IR.

Metadata Issues Metadata vs. full-text search--not either-or situation. Robust metadata industry: publishers (ISI, CA), libraries, archives, museums, A & I Services. Automatic extraction and generation of metadata. Role of XML. Problems with reverse engineering metadata into large unstructured chunks of text. Image retrieval. David Forsyth and Jennifer Trant image database. Google (3 billion—4.2 billion). Open Web with no metadata vs. Hidden Web (500 times larger?) with metadata. Problem areas in full-text searching: authors, subject/author browse, mathematics, precision, Google PageRank. Users trust of Web searches (41%/51%) vs. library search (99%/97%). DLF and Outsell study. Metadata for presentation (MathML). OpenURL and CrossRef and Link Resolvers. Studies of OAI metadata show limitations in implementation, but many have rich metadata (ArX, IOP) for S & D.

Ongoing Investigations Relationship between interoperability models for search and discovery: federated searching (OAI harvested) and broadcast, simultaneous searching of distributed repositories. Not mutually exclusive. OAI Provider and Harvesting software. Rights Management. Static Repositories. Mathematics -- MathML. Reference Linking integration built on OpenURL and DOI. Search Assistant software with simultaneous search, point-of-contact assistance, and remote reference capability.

OpenURL-Based Services Standard for expressing and transmitting metadata. Promise of standardized, normalized search results. Use in Link Resolvers – commercial and local.. Use of CrossRef metadata database to look up DOIs.

Continuing Issues Role of Authors, Academic Institutions, Libraries, Publishers, Abstracting & Indexing Services. Rapid growth of remote access. Disintermediation may affect both Libraries and Publishers. Information as Function not Place. Provide ‘Digital Libraries’ out of digital collections and services. Web Services Evolution of service mechanisms: processing & archiving, search, remote reference, navigation enhancements, link resolving.

4th Generation Information Systems Integration of heterogeneous information resources. Broadcast and Simultaneous Searching of Multiple Resources tailored to user needs. Remote Reference and Instruction (Collaboration software, multimedia, suggestion-based systems). Integration with Learning Management Systems. Software-Aided Search Navigation (Best-Match searching) and Strategy Modification. Dynamic Links to Full-Text. Appropriate Copy problem. One-Stop-Shopping.