PSIgate Knowledge Exchange: Using OAI to Share Information Paul Meehan, PSIgate Technical Manager UKSG Meeting. May 14, 2003

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Subjects to Cover  Google, PSIgate and the RDN  Database Services and Partners  Why OAI?  PSIgate Information Exchange via OAI  OAI Pros and Cons  Future Developments

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 The Trouble with Google  It’s too big! e.g. copper – 5.15 million hits copper + element – 518,000 copper + element + properties – 163,000  relevancy and ranking Only 7 out of first 25 “copper” hits refer to the metal But as a starting point it’s still number 1!!

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 This is what happens …  Lost in (hyper)space!  You lose focus  You waste time  You miss information  FRUSTRATION!

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 The RDN The Resource Discovery Network (RDN) is a collaboration of over seventy educational and research organisations, including the Natural History Museum and the British Library  It is a free service, funded by JISC  It is divided into a number of subject gateways or “hubs”, each of which selects and catalogues Internet resources for learning, teaching and research. Each hub provides search and browse interfaces to its records  The RDN additionally provides a central database of records  Subject gateways may also provide additional services

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 The RDN EEVL: Engineering BIOME: Biomedical sciences Humbul: Humanities SOSIG: Social sciences PSIgate: Physical sciences

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 PSIgate PSIgate is the physical sciences gateway of the RDN Based in Manchester; team of 4 We offer a number of database services including:  Internet Resource Catalogue  astronomy, chemistry, earth sciences, materials, physics, history and policy of science  Expanded “Web Catalogue”  “additional services” We work with a number of professional bodies and organisations e.g. RSChem, IoP, BGS

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 PSIgate vs Google PSIgate IRC*Web Catalogue*Google Copper million Copper + element ,000 Copper + element + properties64 163,000 * IRC contains 7600 resources, Web catalogue ca. 80,000 at May 12, 2003 Catalogued by subject specialists Should all be relevant!

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 PSIgate – Additional Services  News channels  Spotlight science magazine  Reference data  Science timelines  Plug-ins  Training materials  Featured sites and topics  Databases provided by external sources  Partnership with LTSNs … all of these require different technologies … including OAI!

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Why OAI? The Open Archives Initiative (OAI) “develops and promotes interoperability standards that aim to facilitate the efficient dissemination of content” We should aim to use OAI because:  Promotes interoperability  Provides standard tools  Free!  It is supported by an increasing number of data providers

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 PSIgate Information Exchange We currently use OAI for three distinct purposes:  Exposure of our records for gathering by the RDN  Harvesting of records from the IoP  Exposure of records to our LTSN partners Each of these services has its own quirks and methods!

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Using OAI to Share Records with the RDN The RDN gathers resources from all the hubs to provide a central “ResourceFinder” service We store our records in a MySQL database (1), and convert them to “OAI format” on a regular basis (2). The RDN gathers the records (3,4) and stores them in a Cheshire database (5), which is then cross-searched via Z PSIgate Database (MySQL) PSIgate Records (OAI) RDN repository (OAI) Stored in Cheshire Database, search using Z39.50 resources 2 - weekly 3 - request 4 - retrieval 5 1

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Step 1 – Record Conversion Firstly we need to create a series of OAI compliant records from our MySQL database. This is done by running a simple script and creating XML format records. This process is done automatically once per week. A typical record will contains the following core fields …

PSIgate Knowledge Exchange: using OAI to share information IDENTIFIER - 1 TITLE DESCRIPTION RESOURCE TYPE KEYWORD(S) IDENTIFIER - 2 PUBLISHER LANGUAGE RIGHTS oai:psigate.ac.uk: Aroma Chemistry A small set of overheads to accompany a lecture on aroma chemistry, by Terry Acree of Cornell University. The overheads provide theoretical summaries relating to: volatility, activity (Raoult's law), potency, odour units, complexity and mixtures. Lecture notes and courses smells odours en This metadata record is for use by RDN partners only.

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Key points  we create a bulk OAI repository once per week  the RDN is granted access to this repository  the RDN harvest this metadata once per week  we use the latest version (2.0) of the OAI-PMH OAI is efficient for this because:  the records are easy to generate  the process is automated  the RDN handles the retrieval and subsequent storage  The PSIgate database currently stands at around 7,600 resources. The central ResourceFinder service stores more than 50,000 records

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 The IoP Select Service The Institute of Physics Publishing offers a service known as “IOP Select”. “A FREE service from IoP Journals comprising articles chosen by our Editors for their novelty, significance and potential impact on future research”  free  updated weekly  normally requires username and password; direct access via PSIgate

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Using OAI with IoP Select The IoP allow us to gather their “select” resources via OAI  we use a harvesting tool to gather records on a weekly basis (1,2)  we extract relevant fields from each record eg title, abstract, URL (3)  we create search and browse interfaces to the records (4,5) IoP Repository (OAI) PSIgate repository (OAI) 1 - request 2 - retrieval 3 - convert Key fields (text file) Search interface Browse interface 4 5

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Browse all papers Search by keyword

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Option 1 – View all papers

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Option 2 – Keyword search

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Our experience of using OAI with IOP Select Pros:  gathering is simple, using standard OAI-PMH conventions  fast (c. 5 minutes to search and retrieve records from large database)  we store records so we can adapt and create search interfaces easily  maintenance of records occurs at IoP end  building of search interface Cons:  record storage uses disk space (not significant at present, < 1MB)  we need to convert and store records in searchable format  building of search interface (!)

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Using OAI to Share Records with the LTSNs We are engaged in projects with four LTSN partners: LTSN Physical Sciences UK Centre for Materials Education LTSN Geography, Earth and Environmental Science (GEES) LTSN Built Environment OAI allows us to expose our metadata records to these partners … work to be undertaken Summer 2003

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Future Work  LTSN partnerships  Other OAI data providers may offer useful data  Additional work with professional bodies and the IoP  Sharing records with RDN partners and hubs  GEsource

PSIgate Knowledge Exchange: using OAI to share information Paul Meehan UKSG Meeting. May 14, 2003 Conclusions The Open Archives Initiative is already offering PSIgate a significant means of collaborating with external sources:  fast  fairly simple methodology  gaining increasingly widespread usage  harvesting processes can easily be automated  some work to be done with storage and searching