1. OAI Implementation Notes for LTRS, NACA and Open Video Michael L. Nelson NASA Langley Research Center & University of North Carolina mln@ils.unc.edu http://www.ils.unc.edu/~mln/ OAI Open Meeting, Washington DC, January 23, 2001

2. Collections Represented NASA –LTRS (Langley Technical Report Server) ~2300 reports, begun in 1992 http://techreports.larc.nasa.gov/ltrs/ –OAI: http://techreports.larc.nasa.gov/ltrs/oai/ –NACA (National Advisory Committee for Aeronautics) NACA was the predecessor organization to NASA, operating from 1917-1958 ~6300 reports, begun in 1996 http://naca.larc.nasa.gov/ –OAI: http://naca.larc.nasa.gov/oai/

3. Collections Represented University of North Carolina –The Open Video Project ~ 200 public domain video segments, project begun in 1998 http://www.open-video.org/ –OAI: http://buckets.dsi.internet2.edu/openvideo/oai/ Open Video contents and OAI services still strictly experimental

4. NASA: Why is OAI Important? NASA builds DLs out of necessity, but ultimately NASA is a publisher Interested in maximum exposure of and accessibility to its “unrestricted, unlimited” contents In the NASA DLs, we left our “dark matter” partially exposed –individual reports were spidered by robots anyway… –OAI provides a more formal interface & protocol for exposing contents

5. UNC: Why is OAI Important? goal is to grow Open Video into a TREC-like corpus for video segments to share with the research community –a standard collection of short (10 seconds – 1 hour) video segments on which to perform video content based retrieval –variability in video types: color/b&w, sound/no sound, high/low motion, etc. –currently in MPEG-1 others formats in the future

6. OAI Implementation Protocol only specifies CGI stub –many implementations possible I used a “bucket” for each: LTRS, NACA & Open Video –buckets are aggregative, computational entities normally used for data storage generally, 1 bucket per “report” –buckets = metadata + data + methods

7. OAI Bucket Structure Bucket _method.pkg_http.pkg_log.pkg_tc.pkg _md.pkg_state.pkg source files for methods http dependency files logs terms and conditions metadatabucket state default bucket packages bucket payload is DL specific support library index.cgi oai oai.pl element is a support library that defines access for the specific DL in addition to the ~ 30 bucket methods each OAI verb is implemented as a separate method

8. NACA OAI Implementation NACA file system 191719181958... naca-tn-1 refer metadata thumbnail GIFs full size GIFs index.cgi normal WWW use OAI requests OAI Server OAI responses built from examining structure of NACA filesystem LTRS, NACA, Open Video have different file structures, metadata formats,etc.

9. Implementation Did not implement sets –possible set candidates: NACA: years, report type LTRS: NASA STI subject classification Only supporting Dublin Core –DC not sufficient for targeted applications Did not implement resumptionToken

10. 302 Load Balancing Interactive users on main DL machine should not be impacted by metadata harvesting –don’t take deliveries through the front door OAI Server naca.larc.nasa.gov/oai/ if load > 0.05 redirect request OAI Server buckets.dsi.internet2.edu/naca/oai/ harvester http://blah/oai/?verb=ListIdentifiers HTTP Status Code 302 http://blah/oai/?verb=ListIdentifiers … …

11. Metadata Quality XML is very brittle – 1 bad character in the metadata and an entire ListIdentifiers mesg can be damaged –yes, my DLs should be more diligent about scrubbing their metadata, but… –author contributed metadata particularly a problem (e.g. control characters from copy-n-paste) –one advantage of resumptionToken is that it compartmentalizes bad data

12. OAI Impact Can use OAI to build our own generalized services – updates, alerts Finally have a clean method to export metadata, both to: –the general community for unrestricted data –closed communities with restricted data Los Alamos, Air Force Research Laboratory, NASA