Presenter: Yoel Kortick How to set up and use the Publishing Based OAI Data Provider Versions 18 and up Presenter: Yoel Kortick

Copyright Statement All of the information and material inclusive of text, images, logos, product names is either the property of, or used with permission by Ex Libris Ltd. The information may not be distributed, modified, displayed, reproduced – in whole or in part – without the prior written permission of Ex Libris Ltd. TRADEMARKS Ex Libris, the Ex Libris logo, Aleph 500, SFX, SFXIT, MetaLib, DigiTool, Verde, Primo, MetaSearch, MetaIndex and other Ex Libris products and services referenced herein are trademarks of Ex Libris, and may be registered in certain jurisdictions. All other product names, company names, marks and logos referenced may be trademarks of their respective owners. DISCLAIMER The information contained in this document is compiled from various sources and provided on an "AS IS" basis for general information purposes only without any representations, conditions or warranties whether express or implied, including any implied warranties of satisfactory quality, completeness, accuracy or fitness for a particular purpose. Ex Libris, its subsidiaries and related corporations (the "Ex Libris Group") disclaim any and all liability for all use of this information, including losses, damages, claims or expenses any person may incur as a result of the use of this information, even if advised of the possibility of such loss or damage. © Ex Libris Ltd., 2007

About this presentation It is in part based on the document “Publishing Based OAI Data Provider.pdf” which is available on the Ex Libris Doc Portal. It differs from the above document in that it is more of a “How To” document and less of a “theory” document. This presentation does not replace but rather complements the above document. It gives step by step instruction on how the library can set up and use the Publishing Based OAI Data Provider.

The Goal The ultimate goal here is to set up the system so that specific records based on specific parameters such as “update date” and “set” can be retrieved via a URL and synchronized with another system such as DigiTool or PRIMO. The URL which retrieves specific records can then be used by another system to be synchronize or import records. This presentation concentrates on configuration and use in Aleph and not in the system which will ultimately use the URL for synchronization.

Very Brief Introduction The OAI (Open Archive Initiative) Data Provider implements the OAI Protocol Version 2.0. It supports 6 OAI-PMH requests specified by the OAI protocol. For more information about the protocol, refer to the OAI Web site: http://www.openarchives.org This presentation describes the configuration and usage of the OAI Data Provider in Aleph systems that implement Publishing. Refer to the Aleph Publishing documentation for more information about Publishing. Refer to OAI Data Provider for more information on the non Publishing-based OAI Data Provider.

Architecture Publishing-based Aleph OAI Data Provider exposes data prepared by the Aleph Publishing process. The Aleph OAI Data Provider itself is a part of the Aleph www_server. It works in request/response mode. The request is a URL request built according to OAI protocol. The response is in XML format built according to OAI protocol.

Supported Formats Aleph OAI Data Provider has flat sets organization. It supports two formats: marc21 (http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd) oai_dc (http://www.openarchives.org/OAI/2.0/oai_dc.xsd)

5 Steps to set up the OAI Provider First we will list the 5 steps to set up and use the OAI publishing based data provider and then we will explain each one and provide specific examples. Decide which bases from $alephe_tab/tab_base.lng you want to publish and put those bases in column 2 of $data_tab/tab_publish. Define the publishing format for each set in $data_tab/tab_publish Configure the file $alephe_tab/oai/oaipubconf.xml Publish the records Begin retrieving records and other information via a URL

1. Put desired bases in tab_publish table Decide which bases from $alephe_tab/tab_base.lng you want to publish. Put those bases in column 2 of $data_tab/tab_publish. Note: There is no need to publish all of the bases from tab_base.lng, only the ones which you want to publish should be added to tab_publish See example on next slide

1. Put desired bases in tab_publish table $alephe_tab/tab_base.eng !         1          2           3            4     5     6     7   8     9 !!!!!!!!!!!!!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!-!!!!!-!!!!!-!-!!!!!!!!!> USM01                  DEMO USM01 LIBRARY                     USM01 N alldocuments not wst=suppressed YOEL_LIBRARY           The library database USM01             USM01 Y wrd=library YOEL_SCIENCE           The Science database USM01             USM01 Y wrd=science $data_tab/tab_publish !        1                     2          3   4         5 !!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!!!!!!!-!-!!!!!-!!!!!!!!!!!!!!! YOEL_OAI_DC          USM01                N USMDC OAI_DC_XML YOEL_OAI_MARC        USM01                N       OAI_MARC21_XML YOEL_LIBRARY_DC      YOEL_LIBRARY         N USMDC OAI_DC_XML YOEL_LIBRARY_MARC    YOEL_LIBRARY         N       OAI_MARC21_XML YOEL_SCIENCE_DC      YOEL_SCIENCE         N USMDC OAI_DC_XML YOEL_SCIENCE_MARC    YOEL_SCIENCE         N       OAI_MARC21_XML

2. Define publishing format for each set Define the publishing format for each set in $data_tab/tab_publishing It is highly recommended that each set in $data_tab/tab_publish be published in Dublin Core format and MARC21 format. It is not mandatory to publish in both formats. You can publish in marc21 or in oai_dc only. According to OAI standard OAI server must support oai_dc.

2. Define publishing format for each set For Dublin Core format column 5 of tab_publish should be OAI_DC_XML, and for MARC21 format column 5 of tab_publish should be OAI_MARC21_XML In example on next slide from $data_tab/tab_publish the set YOEL_OAI_DC is published in Dublin Core format and the set YOEL_OAI_MARC is published in MARC21 format

2. Define publishing format for each set $data_tab/tab_publish Dublin core format !        1                     2          3   4         5 !!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!!!!!!!-!-!!!!!-!!!!!!!!!!!!!!! YOEL_OAI_DC          USM01                N USMDC OAI_DC_XML YOEL_OAI_MARC        USM01                N       OAI_MARC21_XML MARC21 format Column 4 of tab_publish for publishing set YOEL_OAI_DC uses fix procedure USMDC. USMDC appears in $data_tab/tab_fix as follows: ! 1                 2                              3 !!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!> USMDC fix_doc_convit                 FILE=tab_fix_usm2dc The file tab_fix_usm2dc which appears in column 3 of tab_fix may be used from the default setup to convert USMARC21 to Dublin Core. It appears in $data_tab of bibliographic library

3. Configure file oaipubconf.xml Configure the file $alephe_tab/oai/oaipubconf.xml Three main rules for setting up oaipubconf.xml: The element internalSet must be exactly like it appears in tab_publish The elements setSpec and setName can be whatever user desires (with no spaces in the setSpec). The first set listed should include all of the records which are in the subsequent sets

3. Configure file oaipubconf.xml Thus, if we have tab_publish as it appears in this presentation, we may have the following three set elements in $alephe_tab/oai/oaipubconf.xml: <set>   <setSpec>YOEL</setSpec>   <setName>YOEL Entire College</setName>   <internalSet>YOEL_OAI_DC:oai_dc</internalSet>   <internalSet>YOEL_OAI_MARC:marc21</internalSet> </set>   <setSpec>LIBRARY</setSpec>   <setName>Yoel Library Science</setName>   <internalSet>YOEL_LIBRARY_DC:oai_dc</internalSet>   <internalSet>YOEL_LIBRARY_MARC:marc21</internalSet>   <setSpec>SCIENCE</setSpec>   <setName>Yoel Science</setName>   <internalSet>YOEL_SCIENCE_DC:oai_dc</internalSet>   <internalSet>YOEL_SCIENCE_MARC:marc21</internalSet> The first set listed should include all of the records which are in the subsequent sets setSpec and setName can be whatever the user wants, with no space in the setSpec internalSet names are exactly like they appear in tab_publish

3. Configure file oaipubconf.xml For a full example of the file $alephe_tab/oai/oaipubconf.xml the example supplied with installation. After the set elements are defined (as described in the previous slide) the following appears <repositoryName>ExLibris Repository</repositoryName> <baseURL>http://il-Aleph02:8991/OAI</baseURL> <adminEmail>Yoel.Kortick@exlibrisgroup.com</adminEmail> <description> <oai-identifier xmlns="http://www.openarchives.org/OAI/2.0/oai-identifier" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai-identifier http://www.openarchives.org/OAI/2.0/oai-identifier.xsd"> <scheme>oai</scheme> <repositoryIdentifier>il-Aleph02</repositoryIdentifier> <delimiter>:</delimiter> <sampleIdentifier>oai:il-Aleph02:USM01-000000001</sampleIdentifier> </oai-identifier> </description> </oairoot> We will now discuss the above elements on the following slides

3. Configure file oaipubconf.xml repositoryName: Name of the repository. This element is mandatory. <repositoryName>ExLibris Repository</repositoryName> baseURL: URL used in OAI requests. It must have the following form: http://<Aleph web server host>:<Aleph apache port>/OAI. This element is mandatory. <baseURL>http://il-Aleph02:8991/OAI</baseURL> adminEmail: Email of the repository administrator. This element is mandatory. <adminEmail>Yoel.Kortick@exlibris.co.il</adminEmail>

3. Configure file oaipubconf.xml description: Information about the repository. This element is mandatory. It must contain an <oai-identifier> element with a description of the item identifiers naming policy. It may also contain sub-elements with additional information about the repository. <description> <oai-identifier xmlns="http://www.openarchives.org/OAI/2.0/oai-identifier" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai-identifier http://www.openarchives.org/OAI/2.0/oai-identifier.xsd"> <scheme>oai</scheme> <repositoryIdentifier>il-Aleph02</repositoryIdentifier> <delimiter>:</delimiter> <sampleIdentifier>oai:il-Aleph02:USM01-000000001</sampleIdentifier> </oai-identifier> </description> The sub elements of the description are described in the next slide.

3. Configure file oaipubconf.xml <description> <oai-identifier> xmlns="http://www.openarchives.org/OAI/2.0/oai-identifier" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai-identifier http://www.openarchives.org/OAI/2.0/oai-identifier.xsd"> <scheme>oai</scheme> <repositoryIdentifier>il-Aleph02</repositoryIdentifier> <delimiter>:</delimiter> <sampleIdentifier>oai:il-Aleph02:USM01-000000001</sampleIdentifier> </oai-identifier> </description> <oai-identifier> = a sub element of the <description> element <repositoryIdentifier> = a sub element of the < oai-identifier > element. It must contain the repository ID used as part of OAI identifiers. It must start with a Latin letter. <sampleIdenfitier> = a sub element of the < oai-identifier > element. element must contain a sample identifier. The syntax must be: oai:<repository id>:<physical library>-<doc number>

3. Configure file oaipubconf.xml Restart the WWW server. When the OAI configuration in $alephe_tab/oai is changed, the Web server must be restarted (UTIL W / 3/ 1). Start Servers 0. Exit Procedure 1. WWW Server 3. PC Server 4. Z39 Server 5. Z39 Gate 6. HTTP (Apache) Server (Root Password is Required for HTTPD_PORT<1024) 7. Other Server Enter Number [0]

4. Publish the records Publish the records. In order for the records to be retrieved by the Publishing Based OAI Data Provider they must be published. When records are published a Z00P record is created for each record/set combination. The next slide shows record number 650 in multiple Z00P bases

4. Publish the records Sequence : 001504582 Set : YOEL_OAI_DC Library : USM01 Doc Number : 000000650 Timestamp : 200710230811083 Sequence : 001553443 Set : YOEL_OAI_MARC Timestamp : 200710230815239 Sequence : 001601729 Set : YOEL_LIBRARY_DC Timestamp : 200710230822148 Sequence : 001611468 Set : YOEL_LIBRARY_MARC Library : USM01 Doc Number : 000000650 Timestamp : 200710230824029 Sequence : 001623914 Set : YOEL_SCIENCE_MARC Timestamp : 200710230827063 Sequence : 001621196 Set : YOEL_SCIENCE_DC Timestamp : 200710230825522 The sets here are the sets which appear in column 1 of $data_tab/tab_publish and in the <internalSet> elements of $alephe_tab/oai/oaipubconf.xml

4. Publish the records To publish the records run the service publish_04. This will publish all records specified in the service. Publish_04 is run in order to perform initial extract of Aleph catalog records for publishing purposes. After it is run once all new and updated records will be published via the UTIL E 21 daemon.

5. Retrieve desired records Begin retrieving records and other information via a URL The following slides will show sample URLs Substitute your URL with the URL in the samples, but keep the syntax the same.

5. Retrieve desired records See base (set) definitions from oaipubconf.xml http://il-Aleph02:8991/OAI?verb=ListSets These are the bases from which records can be retrieves in the following slides

5. Retrieve desired records Retrieve information about the repository http://il-Aleph02:8991/OAI?verb=Identify

5. Retrieve desired records View specific record number in specific format Here we will view record number 000049922 in marc21 format http://il-Aleph02:8991/OAI?verb=GetRecord&identifier=oai:il-Aleph02:USM01-000049922&metadataPrefix=marc21

5. Retrieve desired records Here we will view record number 000049923 in Dublin core format http://il-Aleph02:8991/OAI?verb=GetRecord&identifier=oai:il-Aleph02:USM01-000049923&metadataPrefix=oai_dc

5. Retrieve desired records Retrieve everything in the set LIBRARY and get record identifiers http://il-Aleph02:8991/OAI?verb=ListIdentifiers&set=LIBRARY&metadataPrefix=marc21 http://il-Aleph02:8991/OAI?verb=ListIdentifiers&set=LIBRARY&metadataPrefix=oai_dc

5. Retrieve desired records In the following cases we will retrieve records by set and date. The date comes from the Z00P timestamp The timestamp is created for the time in which the Z00P was created (either via publish_04 or UTIL E 21) The set comes from column 1 of tab_publish These sets come from the <setSpec> elements of $alephe_tab/oai/oaipubconf.xml

5. Retrieve desired records We will use a URL consisting of two parts: Part one: http://il-Aleph02:8991/OAI? Part two: verb=ListRecords&from=2007-10-24T00:00:01Z&until=2007-10-24T23:59:59Z&set=LIBRARY&metadataPrefix=marc21 Now we will discuss part two

5. Retrieve desired records verb=ListRecords&from=2007-10-24T00:00:01Z&until=2007-10-24T23:59:59Z&set=LIBRARY&metadataPrefix=marc21 The verb is usually either ListRecords or ListIdentifiers. The examples show what they do. The from and until dates of the time stamp change according to what records the library wishes to retrieve The set changes according to what set (column 1 of tab_publish) the library wishes to retrieve The metadataPrefix is usually either marc21 or oai_dc depending whether the library wants the records in MARC or Dublin Core format.

5. Retrieve desired records Retrieve everything in the set YOEL with Z00P time stamp between minute 2007-10-23T00:00:01 and 2007-10-23T23:59:59. View record identifiers. http://il-Aleph02:8991/OAI?verb=ListIdentifiers&from=2007-10-23T00:00:01Z&until=2007-10-23T23:59:59Z&set=YOEL&metadataPrefix=marc21

5. Retrieve desired records Retrieve everything in the set YOEL with Z00P time stamp between minute 2007-10-23T00:00:01 and 2007-10-23T23:59:59. View record information in marc21 format http://il-Aleph02:8991/OAI?verb=ListRecords&from=2007-10-23T00:00:01Z&until=2007-10-23T23:59:59Z&set=YOEL&metadataPrefix=marc21

5. Retrieve desired records Retrieve everything in the set YOEL with Z00P time stamp between minute 2007-10-23T00:00:01 and 2007-10-23T23:59:59. View record information in Dublin core format http://il-Aleph02:8991/OAI?verb=ListRecords&from=2007-10-23T00:00:01Z&until=2007-10-23T23:59:59Z&set=YOEL&metadataPrefix=oai_dc

5. Retrieve desired records Retrieve everything in the set LIBRARY with Z00P time stamp between minute 2007-10-24T00:00:01 and 2007-10-24T23:59:59. View record information in MARC21 format http://il-Aleph02:8991/OAI?verb=ListRecords&from=2007-10-24T00:00:01Z&until=2007-10-24T23:59:59Z&set=LIBRARY&metadataPrefix=marc21

5. Retrieve desired records If more than approximately thirty records are retrieved then the first approximately thirty will appear on the first web screen of the query Subsequent records will appear after using the resumptionToken element which will appear at the end of the list.

5. Retrieve desired records The resumptionToken is added to the URL as follows: http://il-Aleph02:8991/OAI?verb=ListRecords&resumptionToken=200710230824027200710232359599LIBRARY:YOEL_LIBRARY_MARC

5. Retrieve desired records The remainder (or next approximately 30) of the records will then appear.