1. updated 2003.11.21 10:02 Hong Kong University of Science & Technology Library Workshop on XML-Based Library Applications 4. XML Standards and Tools

2. Hong Kong University of Science & Technology Library Outline XML and Related Standards XML and Related Standards XSLT XSLT OAI-PMH OAI-PMH ZING ZING XML Tools and Software XML Tools and Software XML Database Systems XML Database Systems

3. Hong Kong University of Science & Technology Library XML and Related Standards (1) Categories of XML and Related Standards: Family of standards that facilitate the use of XML documents Family of standards that facilitate the use of XML documents e.g. DOM, XPath, XSL, XSLT, XML Schema, XML Query, XPointer, XLink, XML Encryption, XML Signature, etc. Protocols that allow interchange of data and messages in XML format. Protocols that allow interchange of data and messages in XML format. e.g. SOAP, OAI-PMH, ZING’s SRW, etc.

4. Hong Kong University of Science & Technology Library XML and Related Standards (2) Standards that specify data and metadata in XML format Standards that specify data and metadata in XML format e.g. EAD, RDF, MARC XML, METS, EAC XML, TEI, MathML, NewsML, etc.

5. Hong Kong University of Science & Technology Library XSLT, OAI and ZING XSLT (for Transformation) XSLT (for Transformation) XSL (eX tensible Stylesheet Language) Transformation OAI-PMH (for Harvesting) OAI-PMH (for Harvesting) Open Archives Initiative Protocol for Metadata Harvesting ZING (for Searching) ZING (for Searching) Z39.50 International : Next Generation

6. Hong Kong University of Science & Technology Library What is XSLT? XSL (eXtensible Stylesheet Language) Transformation. XSL (eXtensible Stylesheet Language) Transformation. Developed by W3C for transforming XML documents into other XML documents, using one or more XSLT stylesheets. Developed by W3C for transforming XML documents into other XML documents, using one or more XSLT stylesheets. XSLT stylesheet is itself an XML document. XSLT stylesheet is itself an XML document. It instructs an XSLT Processor on how to transform the XML document. It instructs an XSLT Processor on how to transform the XML document.

7. Hong Kong University of Science & Technology Library XSLT Example (1) Given an XML document: XML documentXML document <name> Smith Smith John John </name>

8. Hong Kong University of Science & Technology Library XSLT Example (2) And you want to transform it to the following XML document: XML document XML document John Smith John Smith

9. Hong Kong University of Science & Technology Library XSLT Example (3) Fragment of an XSLT Stylesheet showing how this transformation can be achieved: XSLT Stylesheet XSLT Stylesheet <person> </person></xsl:template>

10. Hong Kong University of Science & Technology Library XPath To address parts of an XML document To address parts of an XML document Used heavily in XSLT Used heavily in XSLT Example (assume you have a book marked up as an XML document): Example (assume you have a book marked up as an XML document)://chapter /book/chapter/section[title = 'Conclusion'] following-sibling::chapter[position()=1]count(//figure)

11. Hong Kong University of Science & Technology Library XSLT and Web (1) Use XSLT to transform an XML document to HTML (Web page). Use XSLT to transform an XML document to HTML (Web page). Unlike HTML documents, which mix content and page layout in the same document, with XML and XSLT, you can separate content from layout (presentation). Different views of the same content can be presented through the Web via XSLT transformation. Unlike HTML documents, which mix content and page layout in the same document, with XML and XSLT, you can separate content from layout (presentation). Different views of the same content can be presented through the Web via XSLT transformation.

12. Hong Kong University of Science & Technology Library XSLT and Web (2) By applying different XSLT stylesheets to the same XML file, completely different output can be generated: By applying different XSLT stylesheets to the same XML file, completely different output can be generated: Most Wanted Person Most Wanted Person Most Wanted Person Most Wanted Person My Dearest Father My Dearest Father My Dearest Father My Dearest Father

13. Hong Kong University of Science & Technology Library Client/Server Side XSLT Client-side Transformation Client-side Transformation Perform the transformation on the client, e.g. via the web browser, or through client-side scripts. Perform the transformation on the client, e.g. via the web browser, or through client-side scripts. Platform dependent. Platform dependent. Server-side Transformation Server-side Transformation Perform the transformation on the server and transmit the resulting document to the client. Perform the transformation on the server and transmit the resulting document to the client. No need to predict a heterogeneous client platform. No need to predict a heterogeneous client platform.

14. Hong Kong University of Science & Technology Library XSLT Processor Software program to perform transformation based on an XSLT stylesheet. Software program to perform transformation based on an XSLT stylesheet. Many such programs are available for free download from the Internet. See: Many such programs are available for free download from the Internet. See: http://xml.coverpages.org/xslSoftware.html HKUST Library uses Xsltproc of Libxslt HKUST Library uses Xsltproc of Libxslt (http://xmlsoft.org/XSLT/) http://xmlsoft.org/XSLT/

15. Hong Kong University of Science & Technology Library OAI-PMH Open Archives Initiative Protocol for Metadata Harvesting Open Archives Initiative Protocol for Metadata Harvesting An interoperability standard for exposing and harvesting XML-formatted metadata. An interoperability standard for exposing and harvesting XML-formatted metadata. http://www.openarchives.org/ http://www.openarchives.org/ http://www.openarchives.org/ Why OAI? Why OAI? Once OAI-enabled, metadata can be readily harvested Once OAI-enabled, metadata can be readily harvested

16. Hong Kong University of Science & Technology Library OAI Data and Service Providers (1) Data Provider Data Provider exposes metadata about its content exposes metadata about its content e.g. HKUST’s Institutional Repository, XML Name Access Control Repository, DOAJ, etc. e.g. HKUST’s Institutional Repository, XML Name Access Control Repository, DOAJ, etc.DOAJ Service Provider Service Provider harvest metadata from data provider harvest metadata from data provider e.g. OAIster – collects (harvests) freely available, difficult-to-access, academically- oriented digital resources. e.g. OAIster – collects (harvests) freely available, difficult-to-access, academically- oriented digital resources.OAIster

17. Hong Kong University of Science & Technology Library OAI Data and Service Providers (2) Data Provider Service Provider OAI Framework Service Provider XML-Formatted Metadata

18. Hong Kong University of Science & Technology Library OAI Data and Service Providers (3) Data Provider (e.g. HKUST Institutional Repository) OAI Framework Service Provider (e.g. OAIster) XML-Formatted Metadata Data Provider (e.g. HKU Theses Online) Data Provider (e.g. LC American Memory)

19. Hong Kong University of Science & Technology Library OAI-PMH Protocol Requests The five OAI-PMH Protocol Request Commands: The five OAI-PMH Protocol Request Commands: Identify Identify ListMetadataFormats ListMetadataFormats ListIdentifiers ListIdentifiers GetRecord GetRecord ListRecords ListRecords e.g. http://repository.ust.hk/oai/?verb=Identify http://repository.ust.hk/oai/?verb=Identify

20. Hong Kong University of Science & Technology Library OAI-PMH Demonstrations HKUST’s Institutional Repository HKUST’s Institutional Repository Identify, ListMetadataFormats, ListSets, ListRecords, ListIdentifiers, GetRecord Identify, ListMetadataFormats, ListSets, ListRecords, ListIdentifiers, GetRecord IdentifyListMetadataFormatsListSets ListRecordsListIdentifiersGetRecord IdentifyListMetadataFormatsListSets ListRecordsListIdentifiersGetRecord XML Name Access Control Repository XML Name Access Control Repository Identify, ListMetadataFormats, ListSets, ListRecords, ListIdentifiers, GetRecord- MARC21A, GetRecord-MARC21B, GetRecord-EACXML. Identify, ListMetadataFormats, ListSets, ListRecords, ListIdentifiers, GetRecord- MARC21A, GetRecord-MARC21B, GetRecord-EACXML. IdentifyListMetadataFormatsListSetsListIdentifiersGetRecord- MARC21AGetRecord-MARC21B GetRecord-EACXML IdentifyListMetadataFormatsListSetsListIdentifiersGetRecord- MARC21AGetRecord-MARC21B GetRecord-EACXML

21. Hong Kong University of Science & Technology Library ZING Z39.50 International: Next Generation Z39.50 International: Next Generation Covers a collection of initiatives by Z39.50 implementers. Covers a collection of initiatives by Z39.50 implementers. http://www.loc.gov/z3950/agency/zing http://www.loc.gov/z3950/agency/zing http://www.loc.gov/z3950/agency/zing

22. Hong Kong University of Science & Technology Library Why ZING? ZING is an effort to re-deploy Z39.50 in the modern computing environment using Web- and XML-based technologies. ZING is an effort to re-deploy Z39.50 in the modern computing environment using Web- and XML-based technologies. It retains the intellectual contents of Z39.50 and experience gained in the past 20 years of Z39.50 implementations. It retains the intellectual contents of Z39.50 and experience gained in the past 20 years of Z39.50 implementations. 脫胎換骨 脫胎換骨

23. Hong Kong University of Science & Technology Library What is Z39.50? Information retrieval protocol and service (NISO Z39.50-1995 and ISO 23950) Information retrieval protocol and service (NISO Z39.50-1995 and ISO 23950) Allows searching of heterogeneous databases by a common search client Allows searching of heterogeneous databases by a common search client Very popular in the library world and has a 2-decade history of implementation Very popular in the library world and has a 2-decade history of implementation But very little usage outside of the library world But very little usage outside of the library world

24. Hong Kong University of Science & Technology Library Problems of Z39.50 Uses uncommon syntaxes and protocols (OSI-based); incompatible with the modern Web- and XML-based environment Uses uncommon syntaxes and protocols (OSI-based); incompatible with the modern Web- and XML-based environment High learning curve for implementers High learning curve for implementers High implementation cost High implementation cost Not attractive to non-library applications Not attractive to non-library applications Bath Profile ( http://www.nlc-bnc.ca/bath/bp-current.htm ) http://www.nlc-bnc.ca/bath/bp-current.htm

25. Hong Kong University of Science & Technology Library ZING’s SRW SRW - Search and Retrieval for the Web SRW - Search and Retrieval for the Web Retain core functionality of Z39.50 in the form of a SOAP-based Web service Retain core functionality of Z39.50 in the form of a SOAP-based Web service Major concepts in Z39.50 are retained Major concepts in Z39.50 are retained Communication mechanisms Communication mechanisms by SOAP messages by SOAP messages by URL (also known as SRU – Search and Retrieval by URL) by URL (also known as SRU – Search and Retrieval by URL) Search query by: CQL (Common Query Language) Search query by: CQL (Common Query Language)

26. Hong Kong University of Science & Technology Library SRW and SOAP Client SRW SOAP Request: Give me all titles by John Smith Server SRW SOAP Response: No problem, here you are… Request Response Web (HTTP)

27. Hong Kong University of Science & Technology Library SRU Demonstration Search the XML Name Access Control Repository by SRU. Search the XML Name Access Control Repository by SRU. The following URL searches the Repository for exact name 'Ding Yi' : The following URL searches the Repository for exact name 'Ding Yi' : http://lbxml.ust.hk/nac/srw.pl?query=bath.n ame+exact+ding+yi http://lbxml.ust.hk/nac/srw.pl?query=bath.n ame+exact+ding+yi

28. Hong Kong University of Science & Technology Library XML Tools and Databases Free XML Tools and Software Free XML Tools and Software http://www.garshol.priv.no/download/xmltools/ XML and Databases by Ronald Bourret XML and Databases by Ronald Bourret http://www.rpbourret.com/xml/XMLAndDatabases. htm http://www.rpbourret.com/xml/XMLAndDatabases. htm XML Databases Products XML Databases Products http://www.rpbourret.com/xml/XMLDatabaseProds.htm http://www.rpbourret.com/xml/XMLDatabaseProds.htm

29. Hong Kong University of Science & Technology Library XML Database Systems Native XML Database Systems Native XML Database Systems Designed to store XML documents Designed to store XML documents Fundamental unit of storage is XML document Fundamental unit of storage is XML document Examples: Tamino, Berkeley DB XML Examples: Tamino, Berkeley DB XML XML-Enabled Database Systems XML-Enabled Database Systems Any database system that has a gateway to accept XML data, conduct XML-based query, and output XML formatted data. Any database system that has a gateway to accept XML data, conduct XML-based query, and output XML formatted data. Examples: Oracle, Microsoft SQL Server Examples: Oracle, Microsoft SQL Server

30. Hong Kong University of Science & Technology Library Native or not Native? Native XML Databases are most suitable for applications that have: Native XML Databases are most suitable for applications that have: document-oriented data (e.g. full text of a book) document-oriented data (e.g. full text of a book) loosely-structured data with deep nesting (hierarchical) (e.g. bibliographic data) loosely-structured data with deep nesting (hierarchical) (e.g. bibliographic data) data already in XML format (e.g. SOAP messages) data already in XML format (e.g. SOAP messages) Highly structured data fits best with Relational Database Systems (e.g. order record)

31. Hong Kong University of Science & Technology Library Database System Evaluation (1) HKUST Library conducted an evaluation of Native XML Database Systems in March 2001, with an objective to build an XML document storage and retrieval infrastructure for the Library. HKUST Library conducted an evaluation of Native XML Database Systems in March 2001, with an objective to build an XML document storage and retrieval infrastructure for the Library. Tamino from Software AG was selected Tamino from Software AG was selected Most mature and complete at that time Most mature and complete at that time Supports a rich set of programming languages Supports a rich set of programming languages Query by URL Query by URL

32. Hong Kong University of Science & Technology Library Database System Evaluation (2) Software AG is an active participant in XML standards Software AG is an active participant in XML standards Unicode-based, and support major character encodings Unicode-based, and support major character encodings