1. U.S. Government Use of the OAI-PMH Michael L. Nelson Old Dominion University Norfolk Virginia, USA mln@cs.odu.edu http://www.cs.odu.edu/~mln/ ISTEC / NSF Ibero-American Digital Library Joint Project Development Symposium Campinas, Brazil - March 20, 2003

2. Acknowledgements ODU: K. Maly, M. Zubair, J. Bollen, X. Liu LANL: R. Luce, X. Liu NASA: G. Roncaglia, J. Rocker MAGiC (UK): Paul Needham

3. Outline Review of data provider / service provider model –including “aggregators” Role of registration for repositories NASA projects OSTI demo project Technical Report Interchange (TRI) –NASA, DOE, DOD

4. Disclaimer: Scientific and Technical Information (STI) This talk will cover US Government focused / sponsored STI only This talk will not cover American Memory –a cultural history project from the Library of Congress (LoC) http://memory.loc.gov/ –the LoC played a significant role in the definition and early adoption of the OAI-PMH

5. Acronym Review NASADepartment of EnergyDepartment of Defense CASI (Center for AeroSpace Information) http://www.sti.nasa.gov/ OSTI (Office of Scientific and Technical Information) http://www.osti.gov/ DTIC (Defense Technical Information Center) http://www.dtic.mil/ LaRC = Langley Research Center LANL = Los Alamos National Laboratory Sandia = Sandia National Laboratory AFRL = Air Force Research Laboratory

6. Data Providers / Service Providers data providers (repositories) service providers (harvesters)

7. Aggregators data providers (repositories) service providers (harvesters) aggregator aggregators allow for: scalability for OAI-PMH load balancing community building discovery

8. Aggregators Frequently interchangeable terms: –aggregators: likely to be community / institutionally focused –caches: stores a copy, less likely to be community- oriented –proxies: less likely to store a copy, may gateway between OAI-PMH and other protocols Dienst / OAI Gateway; Harrison, Nelson, Zubair, JCDL 03 To learn more about aggregators, caches & proxies: –http://www.openarchives.org/OAI/2.0/guidelines-aggregator.htm –http://www.cs.odu.edu/~mln/jcdl02/

9. Example Aggregators Arc - http://arc.cs.odu.edu/ –first described “hierarchical harvesting” in D- Lib Magazine, 7(4) 2001 http://www.dlib.org/dlib/april01/liu/04liu.html Celestial - http://celestial.eprints.org/ –among other services, it provides a history of harvests (successful vs. errors) http://celestial.eprints.org/cgi-bin/status

10. OAI-PMH 2.0 Registration Data Providers: http://www.openarchives.org/Register/BrowseSites.pl Service Providers: http://www.openarchives.org/service/listproviders.html 75 repositories registered ??? unregistered repositories unregistered because: testing / development not for public harvesting public, but “low-profile” never got around to it… ??? SciELO (> 20k records?) www.scielo.br www.scielo.br/oai/scielo-oai.php DP:SP ~= 5:1

11. Registration is Nice… …But Not Required OAI-PMH is (becoming) the “http” for digital libraries –there is no central registry of http servers remember the NCSA “What’s New” page? (ca. 1994) There will never be “registration support” in OAI- PMH –registries are a type of service provider, built on top of OAI-PMH –registration will be an integral part of community building –friends…

12. A light weight, optional, DP-centric method to communicate the existence of “others” http://techreports.larc.nasa.gov/ltrs/oai2.0/?verb=Identify.. http://naca.larc.nasa.gov/oai2.0 http://ntrs.nasa.gov/oai2.0 http://horus.riacs.edu/perl/oai/ http://ston.jsc.nasa.gov/collections/TRS/oai/..

13. … http://techreports.larc.nasa.gov/ltrs/oai2.0/ http://naca.larc.nasa.gov/oai2.0/ http://ntrs.nasa.gov/oai2.0/ http://ston.jsc.nasa.gov/collections/TRS/oai/ http://horus.riacs.edu/perl/oai/ harvester Identify NASA example

14. Langley Technical Report Server publicly available –began as an anonymous ftp server in 1992; http access in 1993 –model for other technical report servers at other NASA centers details in NASA TM- 109162 mostly LaTeX, MS Word, other systems –some scanned reports http://techreports.larc.nasa.gov/ltrs/ http://techreports.larc.nasa.gov/ltrs/oai2.0/

15. NACA Technical Report Server publicly available –began in 1996 –details in NASA TM-1999- 209127 scanned reports from 1917-1958 –NACA = predecessor to NASA contents mirrored with the MaGIC project –a UK-based grey-literature preservation project –OAI-PMH used to mirror contents http://naca.larc.nasa.gov/ http://naca.larc.nasa.gov/oai2.0/

16. NACA Report 1345 as seen through its native DL http://naca.larc.nasa.gov/

17. NACA Report 1345 as seen through MAGiC http://www.magic.ac.uk/

18. NACA Report 1345 as seen through its Scirus (Elsevier) http://www.scirus.com/

19. NACA Report 1345 as seen through my.OAI (FS Consulting) http://www.myoai.com/

20. NTRS OAI Architecture user... search for “cfd applications” local copy of metadata metadata harvested offline, through OAI interface each node independently maintained individual nodes can still support direct user interaction NTRS LTRSATRSGTRS CASITRS all searching, browsing, etc. performed on the metadata here content (reports) remain archived at the local sites

21. NASA Technical Report Server (nearly) publicly available replacement for the current distributed searching version of NTRS –MySQL –Va Tech harvester –modified “bucket” –details in Nelson, Rocker, Harrison, Library Hi-Tech, 21(2) (March 2003) a service provider & aggregator –same OAI baseURL as used for interactive searching http://ntrs.nasa.gov/

22. NASA Technical Report Server advanced, fielded search explicit query routing –10 NASA repositories –4 non-NASA repositories turned “off” by default

23. non-NASA repositories > 0.5M records

24. NASA DLs in the Larger STI Realm NTRS LTRSATRS CASITRS … DOE DOD UniversitiesPublishers... International NTRS could also be a data provider from the point of view of other DLs; allowing the harvesting of NASA report metadata. NTRS could also harvest metadata from other DLs, and provide access to non-NASA content. We hope to influence the direction of the science.gov effort to use OAI-PMH this could be a fully connected graph

25. OSTI Energy Citations Database OAI-PMH support just recently added (Feb 2003) –not yet officially announced –20k records, 8k full- text other OSTI collections planned http://www.osti.gov/energycitations/

26. Technical Report Interchange Goal: share technical reports between 4 US government labs without creating new digital libraries for users to learn! –NASA Langley Research Center –Air Force Research Laboratory –Los Alamos National Laboratory (DOE) –Sandia National Laboratory (DOE) Solution: use cooperating OAI-PMH caches at each site to –export local contents –ingest remote contents

27. TRI Production System - Status LaRC TRI System LANL TRI System Sandia TRI System AFRL TRI System ODU TRI System (Listener) Records coming in from other TRI systems Records going out to other TRI systems Slide from M. Zubair, ODU Proposed In Production

28. Mappings in TRI Details in Liu, et al. ECDL 2002; the above table also taken from the same paper

29. A Single TRI Module Slide from M. Zubair, ODU

30. The Future: Community Building Ultimately, protocols and metadata formats are not what makes a difference Rather, the critical mass afforded by a common set of utilities (cf. http, Dublin Core, XML) The best current example: The Open Language Archives Community –http://www.language-archives.org/ OAI-PMH provides the basis for communication between strangers, but allows even richer communication between friends

31. STI Communities Government produced/sponsored STI http://ntrs.nasa.gov/ http://www.osti.gov/energycitations/ http://dlib.cs.odu.edu/tri/ Academia –self-archiving vs. institutional archives http://www.soros.org/openaccess/ http://www.ecs.soton.ac.uk/~harnad/Tp/resolution.htm Commercial publishers –e.g. BioMed Central http://www.biomedcentral.com/