1. Presenter: Yoel Kortick
How to set up and use the Publishing Based OAI Data Provider Versions 18 and up
Presenter: Yoel Kortick

2. Copyright Statement
All of the information and material inclusive of text, images, logos, product names is either the property of, or used with permission by Ex Libris Ltd. The information may not be distributed, modified, displayed, reproduced – in whole or in part – without the prior written permission of Ex Libris Ltd.
TRADEMARKS Ex Libris, the Ex Libris logo, Aleph 500, SFX, SFXIT, MetaLib, DigiTool, Verde, Primo, MetaSearch, MetaIndex and other Ex Libris products and services referenced herein are trademarks of Ex Libris, and may be registered in certain jurisdictions. All other product names, company names, marks and logos referenced may be trademarks of their respective owners.
DISCLAIMER The information contained in this document is compiled from various sources and provided on an "AS IS" basis for general information purposes only without any representations, conditions or warranties whether express or implied, including any implied warranties of satisfactory quality, completeness, accuracy or fitness for a particular purpose.
Ex Libris, its subsidiaries and related corporations (the "Ex Libris Group") disclaim any and all liability for all use of this information, including losses, damages, claims or expenses any person may incur as a result of the use of this information, even if advised of the possibility of such loss or damage.
© Ex Libris Ltd., 2007

3. About this presentation
It is in part based on the document “Publishing Based OAI Data Provider.pdf” which is available on the Ex Libris Doc Portal.
It differs from the above document in that it is more of a “How To” document and less of a “theory” document.
This presentation does not replace but rather complements the above document.
It gives step by step instruction on how the library can set up and use the Publishing Based OAI Data Provider.

4. The Goal
The ultimate goal here is to set up the system so that specific records based on specific parameters such as “update date” and “set” can be retrieved via a URL and synchronized with another system such as DigiTool or PRIMO.
The URL which retrieves specific records can then be used by another system to be synchronize or import records.
This presentation concentrates on configuration and use in Aleph and not in the system which will ultimately use the URL for synchronization.

5. Very Brief Introduction
The OAI (Open Archive Initiative) Data Provider implements the OAI Protocol Version 2.0. It supports 6 OAI-PMH requests specified by the OAI protocol.
For more information about the protocol, refer to the OAI Web site:
This presentation describes the configuration and usage of the OAI Data Provider in Aleph systems that implement Publishing.
Refer to the Aleph Publishing documentation for more information about Publishing. Refer to OAI Data Provider for more information on the non Publishing-based OAI Data Provider.

6. Architecture
Publishing-based Aleph OAI Data Provider exposes data prepared by the Aleph Publishing process.
The Aleph OAI Data Provider itself is a part of the Aleph www_server. It works in request/response mode.
The request is a URL request built according to OAI protocol. The response is in XML format built according to OAI protocol.

7. Supported Formats
Aleph OAI Data Provider has flat sets organization. It supports two formats:
marc21 (
oai_dc (

8. 5 Steps to set up the OAI Provider
First we will list the 5 steps to set up and use the OAI publishing based data provider and then we will explain each one and provide specific examples.
Decide which bases from $alephe_tab/tab_base.lng you want to publish and put those bases in column 2 of $data_tab/tab_publish.
Define the publishing format for each set in $data_tab/tab_publish
Configure the file $alephe_tab/oai/oaipubconf.xml
Publish the records
Begin retrieving records and other information via a URL

9. 1. Put desired bases in tab_publish table
Decide which bases from $alephe_tab/tab_base.lng you want to publish. Put those bases in column 2 of $data_tab/tab_publish.
Note: There is no need to publish all of the bases from tab_base.lng, only the ones which you want to publish should be added to tab_publish
See example on next slide

10. 1. Put desired bases in tab_publish table
$alephe_tab/tab_base.eng
!         1          2           3            4     5     6     7   8     9
!!!!!!!!!!!!!!!!!!!!-!-!!!!!!!!!!!!!!!!!!!!-!!!!!-!!!!!-!!!!!-!!!!!-!-!!!!!!!!!>
USM01                  DEMO USM01 LIBRARY                     USM01 N alldocuments not wst=suppressed
YOEL_LIBRARY           The library database USM01             USM01 Y wrd=library
YOEL_SCIENCE           The Science database USM01             USM01 Y wrd=science
$data_tab/tab_publish
!        1                     2          3   4         5
!!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!!!!!!!-!-!!!!!-!!!!!!!!!!!!!!!
YOEL_OAI_DC          USM01                N USMDC OAI_DC_XML
YOEL_OAI_MARC        USM01                N       OAI_MARC21_XML
YOEL_LIBRARY_DC      YOEL_LIBRARY         N USMDC OAI_DC_XML
YOEL_LIBRARY_MARC    YOEL_LIBRARY         N       OAI_MARC21_XML
YOEL_SCIENCE_DC      YOEL_SCIENCE         N USMDC OAI_DC_XML
YOEL_SCIENCE_MARC    YOEL_SCIENCE         N       OAI_MARC21_XML

11. 2. Define publishing format for each set
Define the publishing format for each set in $data_tab/tab_publishing
It is highly recommended that each set in $data_tab/tab_publish be published in Dublin Core format and MARC21 format. It is not mandatory to publish in both formats. You can publish in marc21 or in oai_dc only. According to OAI standard OAI server must support oai_dc.

12. 2. Define publishing format for each set
For Dublin Core format column 5 of tab_publish should be OAI_DC_XML, and for MARC21 format column 5 of tab_publish should be OAI_MARC21_XML
In example on next slide from $data_tab/tab_publish the set YOEL_OAI_DC is published in Dublin Core format and the set YOEL_OAI_MARC is published in MARC21 format

13. 2. Define publishing format for each set
$data_tab/tab_publish
Dublin core format
!        1                     2          3   4         5
!!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!!!!!!!-!-!!!!!-!!!!!!!!!!!!!!!
YOEL_OAI_DC          USM01                N USMDC OAI_DC_XML
YOEL_OAI_MARC        USM01                N       OAI_MARC21_XML
MARC21 format
Column 4 of tab_publish for publishing set YOEL_OAI_DC uses fix procedure USMDC.
USMDC appears in $data_tab/tab_fix as follows:
! 1                 2                              3
!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!-!!!!!!!!!!!!!!!!!!!!!!!!!!!!!>
USMDC fix_doc_convit                 FILE=tab_fix_usm2dc
The file tab_fix_usm2dc which appears in column 3 of tab_fix may be used from the default setup to convert USMARC21 to Dublin Core. It appears in $data_tab of bibliographic library

14. 3. Configure file oaipubconf.xml
Configure the file $alephe_tab/oai/oaipubconf.xml
Three main rules for setting up oaipubconf.xml:
The element internalSet must be exactly like it appears in tab_publish
The elements setSpec and setName can be whatever user desires (with no spaces in the setSpec).
The first set listed should include all of the records which are in the subsequent sets

15. 3. Configure file oaipubconf.xml
Thus, if we have tab_publish as it appears in this presentation, we may have the following three set elements in $alephe_tab/oai/oaipubconf.xml:
<set>
  <setSpec>YOEL</setSpec>
  <setName>YOEL Entire College</setName>
  <internalSet>YOEL_OAI_DC:oai_dc</internalSet>
  <internalSet>YOEL_OAI_MARC:marc21</internalSet>
</set>
  <setSpec>LIBRARY</setSpec>
  <setName>Yoel Library Science</setName>
  <internalSet>YOEL_LIBRARY_DC:oai_dc</internalSet>
  <internalSet>YOEL_LIBRARY_MARC:marc21</internalSet>
  <setSpec>SCIENCE</setSpec>
  <setName>Yoel Science</setName>
  <internalSet>YOEL_SCIENCE_DC:oai_dc</internalSet>
  <internalSet>YOEL_SCIENCE_MARC:marc21</internalSet>
The first set listed should include all of the records which are in the subsequent sets
setSpec and setName can be whatever the user wants, with no space in the setSpec
internalSet names are exactly like they appear in tab_publish

16. 3. Configure file oaipubconf.xml
For a full example of the file $alephe_tab/oai/oaipubconf.xml the example supplied with installation.
After the set elements are defined (as described in the previous slide) the following appears
<repositoryName>ExLibris Repository</repositoryName>
<baseURL>
<description>
<oai-identifier
xmlns="
xmlns:xsi="
xsi:schemaLocation="
<scheme>oai</scheme>
<repositoryIdentifier>il-Aleph02</repositoryIdentifier>
<delimiter>:</delimiter>
<sampleIdentifier>oai:il-Aleph02:USM </sampleIdentifier>
</oai-identifier>
</description>
</oairoot>
We will now discuss the above elements on the following slides

17. 3. Configure file oaipubconf.xml
repositoryName: Name of the repository. This element is mandatory.
<repositoryName>ExLibris Repository</repositoryName>
baseURL: URL used in OAI requests. It must have the following form:
web server host>:<Aleph apache port>/OAI.
This element is mandatory.
<baseURL>
admin of the repository administrator. This element is mandatory.

18. 3. Configure file oaipubconf.xml
description: Information about the repository. This element is mandatory. It must contain an <oai-identifier> element with a description of the item identifiers naming policy. It may also contain sub-elements with additional information about the repository.
<description>
<oai-identifier
xmlns="
xmlns:xsi="
xsi:schemaLocation="
<scheme>oai</scheme>
<repositoryIdentifier>il-Aleph02</repositoryIdentifier>
<delimiter>:</delimiter>
<sampleIdentifier>oai:il-Aleph02:USM </sampleIdentifier>
</oai-identifier>
</description>
The sub elements of the description are described in the next slide.

19. 3. Configure file oaipubconf.xml
<description>
<oai-identifier>
xmlns="
xmlns:xsi="
xsi:schemaLocation="
<scheme>oai</scheme>
<repositoryIdentifier>il-Aleph02</repositoryIdentifier>
<delimiter>:</delimiter>
<sampleIdentifier>oai:il-Aleph02:USM </sampleIdentifier>
</oai-identifier>
</description>
<oai-identifier> = a sub element of the <description> element
<repositoryIdentifier> = a sub element of the < oai-identifier > element. It must contain the repository ID used as part of OAI identifiers. It must start with a Latin letter.
<sampleIdenfitier> = a sub element of the < oai-identifier > element. element must contain a sample identifier. The syntax must be: oai:<repository id>:<physical library>-<doc number>

20. 3. Configure file oaipubconf.xml
Restart the WWW server.
When the OAI configuration in $alephe_tab/oai is changed, the Web server must be restarted (UTIL W / 3/ 1).
Start Servers
0. Exit Procedure
1. WWW Server
3. PC Server
4. Z39 Server
5. Z39 Gate
6. HTTP (Apache) Server (Root Password is Required for HTTPD_PORT<1024)
7. Other Server
Enter Number [0]

21. 4. Publish the records Publish the records.
In order for the records to be retrieved by the Publishing Based OAI Data Provider they must be published. When records are published a Z00P record is created for each record/set combination.
The next slide shows record number 650 in multiple Z00P bases

22. 4. Publish the records
Sequence :
Set : YOEL_OAI_DC
Library : USM01
Doc Number :
Timestamp :
Sequence :
Set : YOEL_OAI_MARC
Timestamp :
Sequence :
Set : YOEL_LIBRARY_DC
Timestamp :
Sequence :
Set : YOEL_LIBRARY_MARC
Library : USM01
Doc Number :
Timestamp :
Sequence :
Set : YOEL_SCIENCE_MARC
Timestamp :
Sequence :
Set : YOEL_SCIENCE_DC
Timestamp :
The sets here are the sets which appear in column 1 of $data_tab/tab_publish and in the <internalSet> elements of $alephe_tab/oai/oaipubconf.xml

23. 4. Publish the records
To publish the records run the service publish_04. This will publish all records specified in the service.
Publish_04 is run in order to perform initial extract of Aleph catalog records for publishing purposes.
After it is run once all new and updated records will be published via the UTIL E 21 daemon.

24. 5. Retrieve desired records
Begin retrieving records and other information via a URL
The following slides will show sample URLs
Substitute your URL with the URL in the samples, but keep the syntax the same.

25. 5. Retrieve desired records
See base (set) definitions from oaipubconf.xml
These are the bases from which records can be retrieves in the following slides

26. 5. Retrieve desired records
Retrieve information about the repository

27. 5. Retrieve desired records
View specific record number in specific format
Here we will view record number in marc21 format

28. 5. Retrieve desired records
Here we will view record number in Dublin core format

29. 5. Retrieve desired records
Retrieve everything in the set LIBRARY and get record identifiers

30. 5. Retrieve desired records
In the following cases we will retrieve records by set and date.
The date comes from the Z00P timestamp
The timestamp is created for the time in which the Z00P was created (either via publish_04 or UTIL E 21)
The set comes from column 1 of tab_publish
These sets come from the <setSpec> elements of $alephe_tab/oai/oaipubconf.xml

31. 5. Retrieve desired records
We will use a URL consisting of two parts:
Part one:
Part two: verb=ListRecords&from= T00:00:01Z&until= T23:59:59Z&set=LIBRARY&metadataPrefix=marc21
Now we will discuss part two

32. 5. Retrieve desired records
verb=ListRecords&from= T00:00:01Z&until= T23:59:59Z&set=LIBRARY&metadataPrefix=marc21
The verb is usually either ListRecords or ListIdentifiers. The examples show what they do.
The from and until dates of the time stamp change according to what records the library wishes to retrieve
The set changes according to what set (column 1 of tab_publish) the library wishes to retrieve
The metadataPrefix is usually either marc21 or oai_dc depending whether the library wants the records in MARC or Dublin Core format.

33. 5. Retrieve desired records
Retrieve everything in the set YOEL with Z00P time stamp between minute T00:00:01 and T23:59:59. View record identifiers.

34. 5. Retrieve desired records
Retrieve everything in the set YOEL with Z00P time stamp between minute T00:00:01 and T23:59:59. View record information in marc21 format

35. 5. Retrieve desired records
Retrieve everything in the set YOEL with Z00P time stamp between minute T00:00:01 and T23:59:59. View record information in Dublin core format

36. 5. Retrieve desired records
Retrieve everything in the set LIBRARY with Z00P time stamp between minute T00:00:01 and T23:59:59. View record information in MARC21 format

37. 5. Retrieve desired records
If more than approximately thirty records are retrieved then the first approximately thirty will appear on the first web screen of the query
Subsequent records will appear after using the resumptionToken element which will appear at the end of the list.

38. 5. Retrieve desired records
The resumptionToken is added to the URL as follows:

39. 5. Retrieve desired records
The remainder (or next approximately 30) of the records will then appear.