1. OAIster: A One-Stop-Shop Service for Digital Objects Kat Hagedorn OAIster Librarian University of Michigan Libraries September 18, 2003

2. background One-year Mellon grant project to test the feasibility of making OAI-enabled metadata for digital objects accessible to the public Digital Library Production Service at University of Michigan Libraries began work in December 2001 Publicized as OAIster in February 2002 Launched in June 2002

3. highlights Any audience Any subject matter Any format Freely accessible No dead ends One-stop shopping …retrieving the “hidden web”

4. the protocol OAI = Open Archives Initiative OAI-PMH = Open Archives Initiative Protocol for Metadata Harvesting Designed to make it easy to exchange metadata among interested parties Consists of 6 HTTP requests to identify repositories / metadata and perform “harvesting”

5. tools we used University of Illinois Urbana-Champaign open-source OAI protocol harvester Our own Digital Library Extension Service (DLXS) environment for building and mounting digital libraries A java tool for processing the harvested records and making them available to our search engine

6. system design UIUC harvester Record storage XSLT transformation tool BibClass indexes OAI-enabled DC records Non-OAI- enabled DC records XSL stylesheets (per source type) Search interface (XPAT)

7. result One place to look for digital objects Big –1,538,431 metadata records –197 institutions (as of August ‘03) Popular –Averages 3300 search sessions / month –Picked up in March ‘03: average 3650 now –43,894 searches total (through July ‘03)

8. www.oaister.org: search

9. www.oaister.org: limiters

10. www.oaister.org: sort

11. www.oaister.org: results

12. www.oaister.org: repositories

13. repositories: e.g., –Online Archive of California: manuscripts, photographs, and works of art held in institutions across California –arXiv Eprint Archive: math and physics pre- and post-prints –Sammelpunkt, Elektronisch Archivierte Theorie: archive of philosophical publications –British Women Romantic Poets Project: collection of poems written by British women between 1789 and 1832

14. repositories: stats As of July ‘03, out of 191 repositories… U.S. and foreign –U.S.: 49% (94) –Foreign: 51% (97) By subject –Humanities: 26% (50) –Science: 30% (58) –Mixed: 43% (83) E-prints and pre-prints –Using eprints.org software: 41% (78) –Not using eprints.org software: 58% (110)

15. major issues encountered Metadata variation Records not leading to digital objects Access restrictions on digital objects described in records Duplicate records for a single digital object

16. issue: metadata variation With more records, users need more restrictions when they search Consistent metadata needed to facilitate these restrictions One option: normalization of data

17. issue: metadata variation Type: the obvious quick win –240 metadata values mapped to four generic values (text, image, audio, video) –e.g., audio, sound = audio motion, animation, newsreels, etc. = video watercolour, watercolor, slides, etc. = image article, articles, booklet, diss, story, etc. = text

18. issue: metadata variation Date: where to begin? –Most records with at least one date –Some records include up to seven dates –No consistent style of date Subject: out of context, what meaning? –Many records with at least one subject element –But over 100 records with more than 50 subjects –And one record with 1000!

19. issue: metadata variation Sample date values 2-12-01 2002-01-01 0000-00-00 1822 between 1827 and 1833 18--? November 13, 1947 SEP 1958 235 bce Summer, 1948

20. issue: metadata variation Sample subject values 30,51,52 1852, Apr. 22. E[veritt] Judson, letter to Philuta [Judson]. Slavery--United States--Controversial literature view of interior with John Henry sculpture Particles (Nuclear physics) -- Research.

21. issue: no digital objects Some records contain links to a further description of a digital object …but not the digital object itself Culling difficult One option: add explanatory text to site

22. issue: access restrictions No records where metadata itself is restricted in use (as far as we know!) Definitely some records where objects are restricted to licensed users One option: add explanatory text to site

23. issue: access restrictions DC Rights element: often not enough info about viewing restrictions Currently no protocol method for indicating restricted digital objects (i.e., “yes/no” toggle element) Need to assess whether users feel informed or frustrated when encountering restricted objects

24. issue: duplicate records Two records harvested, different identifiers, same object described and pointed to Acquired in two ways: –Harvesting of original repository and aggregator –Receiving “static” DC records provided by content creator and harvesting aggregator

25. issue: duplicate records Aggregators can contain records not currently available through OAI channels Aggregators do not always contain all the records of a particular original repository So, need to harvest both aggregator and original repositories

26. issue: duplicate records Harvest records from aggregator Also receive from original content creator, but as snapshot –e.g., MEO and cogprints –Snapshot before aggregator –Creator unsure all records would be aggregated

27. issue: duplicate records Were duplicates to be identified, how to deal with the issue? –Suppress? –Group? –Flag? So far, not addressed in OAIster

28. assessment Large survey (over 400 respondents) 2 rounds of face-to-face and remote user testing Conducted before design and after phase one rollout

29. assessment: survey Online journals and reference materials wanted over other digital objects Difficult to search for information; every service different; where to start Number of respondents (5%) indicated they were generally successful in finding resources online

30. assessment: user testing No short and long record formats: one size fits all Want clearly defined and labeled AND/OR searching options Results clear and easy to understand Want to sort by title, date, institution, resource format…you name it! Use OAIster for academic, trustworthy, authentic materials

31. service providers: comparison Focus on high usability Focus on all content available Some service providers have increased functionality (e.g., de- duplication, integration of thesauri) DP-9 OAIster Ad hoc Usability Content someall high low UIUC, Emory, etc.

32. future of OAIster Make it faster Advanced searching Grouping to aid browsing Saving/emailing/downloading records Further normalization of data Handling duplicate records Collaboration with other services: search, instructional…

33. current state of protocol Popular As Peter Suber says: –“…no other single idea or technology in the [open- source movement] has enjoyed this density of endorsement and adoption in a six month period.” Data providers over one year: –June ‘02: 56 repositories / 274,062 records –June ‘03: 187 repositories / 1,246,953 records –Over three-fold increase for repositories –Over four-fold increase for records

34. can you be in OAIster? Yes! Just OAI-enable your data –Designed to be simple to implement –Make sure data complies with protocol and schema –Provide as much metadata as you can –Develop “sets” for service providers Let us know you’re ready to be harvested Keep us informed about changes to the harvesting URL, new data and deleted data, change in contact info

35. how can you use OAIster? Just about anywhere… Reference desks Tool for researchers and faculty Inclusion into list of electronic resources and/or subject guides It is: –freely available –regularly updated –simple to use

36. contact info Kat Hagedorn University of Michigan Libraries, Digital Library Production Service khage@umich.edu http://www.oaister.org/