An Overview of Selected ISO Standards Applicable to Digital Archives Science Archives in the 21st Century 25 April 2007 Donald Sawyer - NASA/GSFC/NSSDC Lou Reich - CSC (NASA/GSFC) John Garrett - Raytheon (NASA/GSFC/NSSDC)

Overview NSSDC Standards Participation: Historical Perspective Selected ISO standards for Archives –Reference Model of an Archive and its Information –Checklist of Activities between Data Providers and Archives –Packaging Data and Metadata with an XML Manifest –Describing Data and Sending it to an Archive –Ensuring Archives can be Trusted to Preserve Information Wrap-up Views

NSSDC Standards Participation: Historical Perspective NSSDC formed NASA/Science Office of Standards and Technology (NOST) in 1988 –Created and supported a formal NOST standards development process –NOST FITS Technical Committee formed by Astrophysics discipline –In 1999, NOST Definition of FITS: NOST NSSDC began participation in Consultative Committee for Space Data Systems (CCSDS) standards development in 1986 –Data packaging standard (SFDU): ISO –Data description registration authority procedures: ISO –Data description packaging for registration: ISO –PVL data tagging language: ISO –Data description language EAST specification: ISO –Data entity dictionary specification: ISO 21961, 21962, NSSDC began formally leading Space Physics Archive Search and Extract (SPASE) development for Heliophysics in 2003 –Version is available for data producers at group.org/data/doc/spase-1_1_0.pdf

Reference Model of an Archive Reference Model for an Open Archival Information System (OAIS): ISO –Standardizes terms, concepts, and basic responsibilities –Addresses functions, information models, migration, federation –Basis for this workshop’s Framework document –Extremely widely adopted Recognized and used throughout virtually all national archives and digital libraries, and many science data repositories, commercial organizations NSSDC uses it for internal and external communication

NSSDC Uses OAIS Concepts Archive Information PackagesProvenance Information Archival Information CollectionsContext Information Archive Information UnitsReference Information Submission Information PackagesFixity Information Content InformationDesignated Community Representation Information

Checklist of Activities between Data Providers and Archives Producer-Archive Interface Methodology Abstract Standard (PAIMAS): ISO Data ready to archive Preliminary Agreement Dictionary Formal model Submission Agreement Transferred object files Validation agreement Phase objective Preliminary Phase Formal Definition Phase Transfer Phase Validation Phase Anomalies Validate the transferred objects Define the information to be archived Develop agreement (data to be delivered, complementary elements, schedule) Actual transfer of the objects in SIPs

Methodology: Preliminary Phase Sub-phases and Actions Action table Description

Packaging Data and Metadata with an XML Manifest XML Formatted Data Unit (XFDU): CCSDS R-1 –Undergoing formal Space Agency review –ISO review to follow –Supports single file implementation (e.g., zip) Contains or points to other files Supports OAIS or custom metadata categorization, mime types, checksums, and transformations such as compression and encryption. Validation API supports a range of structural and semantic validation capabilities Prototype java software library available from NASA –Prototype C++ API available from ESA Current scalability testing to 50 Gbytes

XDFU Environment View

XFDU Manifest Logical View

Describing Data and Sending it to an Archive Producer-Archive Interface Specification (PAIS): CCSDS/ISO Data Archive Ingest WG draft –Formal Space Agency review late summer –ISO review 2008 –Standardized and extensible attributes to describe data objects prior to transfer to archive –Maps data and PAIS information to XFDU packaging standard for transfer Will leverage and extend XFDU software library

PAIS Process Flow

PAIS Formal Descriptions

Ensuring Archives can be Trusted to Preserve Information Digital Repository Audit and Certification: CCSDS/ISO BOF working group –Looking to become chartered CCSDS/ISO Working Group –Build on work completed in other organizations, particularly: Research Libraries Group/National Archives and Records Administration’s “Trustworthy Repositories Audit & Certification: Criteria and Checklist (TRAC)” Network of Expertise in long-term STORage Working Group’s “Catalogue of Criteria for Trusted Digital Repositories” –Subject of subsequent presentation at this workshop

Wrap-up Views Difficult to pre-judge the success of any particular standards effort –May take several years to ‘catch on’ or be superceded However, OAIS reference model ‘caught on’ in draft stage - large need and good timing! –Significant educational component associated with any standards effort Another form of research Contact presenter for more information