1. 1 Schema Registries Steven Hughes, Lou Reich, Dan Crichton NASA 21 October 2015

2. 2 Topics F Background/Definitions F Schema Registry Requirements F Plan

3. 3 Definitions F Registry - Formal record of information, or the place where such records are kept. F Repository - A repository is a place where data is stored and maintained. F Registry and Repository are referenced in the Information Architecture and OAIS RM papers but they are not defined.

4. 4 Registry vs Repository ä Repository holds data F Provides “get” and “put” functions to retrieve and store data holdings using a “unique identifier”. ä Some additional “operative” metadata may exist primarily for managing the data in the repository. (dates, checksums, etc). ä Repository may hold “archive” metadata, but it is co-located with the data. In the repository context the metadata is not considered operative. ä Repository has a “register function”. (i.e. A repository is a register in the sense that a formal record of its contents is kept by registering or “making or securing an official entry”) ä Repository is similar to the “OAIS Archival Storage” functional entity

5. 5 Registry vs Repository ä Registry assigns an identifier to object ä Registry uses metadata to build a catalog (or index) F Makes the metadata searchable for identifying and locating stored objects F Supports two phased data access (i.e. search and then retrieve) ä Registry is similar to the OAIS functional entity “Data Management”. It supports “OAIS Access”. ä Registry is considered separately from a repository since the two functional entities may be deployed separately. ä Registry and Repository functions are often collocated.

6. 6 Types of Registries F Metadata registry provides an interface to search for data about metadata. ä Manage meta-metadata models. (e.g. XML Schemas, Data Dictionaries, Ontologies F Resource registry provides an interface to search for resources. ä Manage descriptions of any resource (i.e. object), typically information objects such as science data products and data sets. F Service registry provides an interface to search for functional services that perform a needed action specified by a user. ä Manage descriptions of service interfaces (called service descriptions), including their respective locations, methods and method parameters.

7. 7 Schema Registry F Repository/registry that accepts, stores, and maintains XML schemas ä Functionality is shared across a distributed environment ä Support schema design, development, validation, deployment, evolution, and maintance

8. 8 XFDU Producer Application XFDU Consumer Application XML Toolkit Library Packaging Plug-ins Transformation Plug-ins Validation Plug-ins Plugin Registry/ Repository XML Registry/ Repository System Programmer ProducerConsumer CCSDS XFDU Administration Registration Dissemination Notification Validation Query Subscription Reconstitution Application Domain Programmer

9. 9 Functional Requirements Full Standards Compliance ä Import and validate W3C XML Schema or DTD F Version Control ä Schema level ä Object level ä Configuration management with audit trails ä Check-In, Check Out ä Release Workflow – local updates, checkin, integration, create release, deploy release ä Subscribe/Notification F Impact Analysis ä Track dependencies between objects ä Determine impact of changes

10. 10 Functional Requirements F Object Management ä Manages schemas at their object level, not at the file or file fragment level (i.e. To the attribute or element level) ä Any redundancy and duplication is resolved at import/update time ä Reuse happens through reference links ä Assemble schemas dynamically from the objects F Common Repository ä Developers work in their own personal workspaces ä Shared object database provides for team-based development

11. 11 Functional Requirements F Documentation ä Object annotation ä Documentation production on request F Multiple namespaces ä Relationships between elements ä Automatic transformations F Schema artifact respository CortexML - Collaborative XML Schema Management http://www.digitalml.com/en/index.php

12. 12 Functional Requirements F Schema Validation ä Validate well formed XML and log errors. F XML Instance Validation ä Validate an XML file against an internally referenced DTD or XSD

13. 13 Plan F Determine interest F Consider working group for schema registries F Gather requirements for schema registries F Define standard information model and services for schema registries F Identify role of schema registry in information infrastructure

14. 14 Backup

15. 15 DSMS Needs Registry Service Components F Data Element/Data Type Registry: Provides common data dictionary information that contains element definitions and data type definitions F XML Schema Registry: Provides a versioned repository for XML schemas that describe messages, file formats, and data components. F XML Stylesheet Registry: Provides a versioned repository for XML stylesheets that perform common data translations. F Namespace/Domain Registry: Provides a controlled, hierarchical namespace/domain registry. F Service Registry: Provides a dynamic Registry for Web Services. F Model Registry: Provides a repository for information models, data relationships, and other ontological information.

16. 16 DSMS Needs CIO/Standards Efforts F Schema Registry ä A component of the Metadata Service. ä Currently deployed in the DSMS development environment. F Data Element Registry ä A component of the Metadata Service. ä Currently deployed in the DSMS development environment. F Service Registry ä Evaluating COTS and Open Source products.

17. 17 Schema Registry Overview F This registry provides a central repository for developers and organizations to publicly submit, publish and exchange XML-based schemas. F A component of the Metadata Service. An effort currently funded by the CIO. F Interfaces ä Application Program Interface (API) F Support distributed access using CORBA/RMI F Support Java and HTTP application interfaces ä Command-Line ä Web-Based GUI F Planned Features for Future Releases ä Support for Authentication and Authorization

18. 18 Schema Registry Current Deployment F Deployed with Metadata Service release 2.0.0 ä http://metadata.jpl.nasa.gov/ on February 9 http://metadata.jpl.nasa.gov/ ä http://bayamo.jpl.nasa.gov:8080/ on March 5 (DSMS Deployment) http://bayamo.jpl.nasa.gov:8080/ F Features ä Add, remove and modify schemas and their supporting metadata in the registry. ä Metadata capture based on the Dublin Core Element Set. ä Allow for versioning and promotion of schemas. ä Capability to locate and access registered schemas. ä Provide for URL access to registered schemas. ä Provide for a web directory hierarchy for organizing schemas.

19. 19 Data Element Registry Overview F This registry provides the capability to register and manage a data element and associate it with a data dictionary. F A component of the Metadata Service. An effort currently funded by the CIO. F Interfaces ä Application Program Interface (API) F Support distributed access using CORBA/RMI F Support Java and HTTP application interfaces ä Command-Line ä Web-Based GUI F Planned Features for Future Releases ä Support for value domain ä Support for element relationships ä Support for Authentication and Authorization

20. 20 Data Element Registry Current Deployment F Deployed with Metadata Service release 2.0.0 ä http://metadata.jpl.nasa.gov/ on February 9 http://metadata.jpl.nasa.gov/ ä http://bayamo.jpl.nasa.gov:8080/ on March 5 (DSMS Deployment) http://bayamo.jpl.nasa.gov:8080/ F Features ä Based on the ISO/IEC 11179 “Information technology -- Specification and standardization of data elements”. ä Add, remove and modify data elements and their supporting attributes in the registry. ä Allow for versioning of data elements. ä Associated data elements to a data dictionary. ä Extract data elements from an imported data dictionary via the Dictionary Registry component. ä Capability to locate and access registered data elements.