Australasian Digital Recordkeeping Initiative – Adrian Cunningham

SESSION OUTLINE Challenge – Making, Keeping and Using Digital Records over long term Australasian Digital Recordkeeping Initiative (ADRI) – objectives, principles Uniform Australasian approach ADRI projects Outstanding issues

CHANGES IN THE RECORD KEEPING ENVIRONMENT Devolution of responsibilities to individual agencies Changes in work processes Internet & data exchange New public records laws Over the past 10 years the environment in which Australasian jurisdictions have been confronting these problems has also undergone marked change. The devolution of responsibilities for recordkeeping from central service agencies to individual program agencies has continued throughout the period. However, on the other side of the coin, in most jurisdictions the recognition that archival authorities should have responsibility for establishing and monitoring standards for recordkeeping has increased considerably, and in many jurisdictions has been formally legislated. At the same time, the development of the Australian Records Management Standard, and several standards for recordkeeping metadata, has provided a common language and framework in which a great deal of useful progress has been achieved in both the public and private sectors. These radical changes have been matched by – indeed surpassed by – the rapid expansion of use of the Internet and the development of technologies for the reliable and authenticated exchange of data in business transactions. So the environment in which we are now confronting the central problem of preservation of digital records is both more challenging and also more conducive to resolution of the main issues.

THE PRESERVATION PROBLEM Rapid obsolescence of Data formats Digital media, densities etc. Hardware Software – operating systems & applications Physical instability of digital media The nature of the challenges for preserving long-term access to digital records has been recognised for over a decade. The logical and physical structuring of data formats in various common proprietary applications has exhibited a norm of changing with each version release; The physical structure, shape, density and operating technology of recording media for digital data has also shown virtually continuous change since the 1960s. The same can be said for computer hardware and operating systems, and for applications software. None of this is news to anyone involved either in industry or government. Many more recent forms of digital recording media are actually even less stable than some of their predecessors – eg., recordable CDs (dye sublimation process)

CONVERGENCE TO XML? International standardisation vs proprietary formats Extensibility = adaptability Platform neutral

AUSTRALIA’S TRACK RECORD The Records Continuum/AS 4390 (1996) E-Permanence/DIRKS/Functional analysis Metadata standards Work Process Analysis for Records, AS 5090 Victorian Electronic Records Strategy National Archives of Australia’s Digital Preservation Program – Xena, etc History of cross-jurisdictional cooperation within a federal system of government

Members since 2004 National Archives of Australia Archives New Zealand Public Record Office Victoria State Records NSW ACT Territory Records Archives Office of Tasmania Northern Territory Archives Service Queensland State Archives State Records South Australia State Records Office Western Australia

ADRI Vision All governments in Australia and New Zealand implement a uniform approach to making, keeping and using digital records to ensure accountability and the long-term protection of vital cultural heritage. This uniformity leads to efficiency, economy and interoperability across participating communities.

ADRI Objectives (1) That all ADRI members are committed to agreed principles for digital recordkeeping. That ADRI supports members in advocating for appropriate digital recordkeeping to support government in their jurisdictions. That each government in Australia and New Zealand has an e-governance regime which is supported by sound strategies for making, keeping and using digital records. That vendors provide implementations of standards developed by ADRI.

ADRI Objectives (2) That the evidence of e-government of Australia and NZ is captured, preserved and accessible for the benefit of current and future generations. That professional leadership and development is provided in digital recordkeeping for Australia and New Zealand. That, in a digital environment, agencies can meet their legal and functional responsibilities effectively and economically. That the best possible strategic use of limited collective resources is made.

ADRI Guiding Principles (1) 1. Advocacy and assistance responsibilities of government archives. 2. Primary importance of the design and implementation of recordkeeping systems.  3. Shared rights and responsibilities and the need for adequate support and training. 4. Commitment to common standards. 5. Importance of identifying recordkeeping requirements. 6. Importance of standardised recordkeeping metadata.

ADRI Guiding Principles (2) 7. Records that are made in digital form should be retained in digital form. Digital records need to be actively managed and maintained. Preservation formats should not be constrained by proprietary rights. Need for trusted digital repositories. Public access should be maximised by full use of available networked technologies.

Uniform Australasian Approach Making & Managing Digital Records Keeping Digital Records (both in agencies and in the archives) Transferring Digital Records to Archives Using Digital Records

Making & Managing Digital Records Guidelines and tools for: ·   Creating accurate & reliable records ·   Classification and control metadata for records ·   Technology-specific issues for records (eg., email; encryption; web-based resources) ·   Functional requirements for r/k systems ·   Model software procurement specifications ·   Compliance assessment and/or self-diagnosis ·   Schemas for record genres Standards for: ·    Recordkeeping metadata ·    Methodologies for recordkeeping system design and implementation

Keeping Digital Records (both in agencies and in the archives) Guidelines on: ·   Preferred long term data formats ·   XML–based approaches to digital preservation ·   Migration paths for long-term digital records ·   Preservation treatment of specific data formats (e.g. database-derived records; web records) ·   Process models for preservation of digital records in agency custody ·   Dealing with changes in the machinery of government over time ·   Process models & recommendations for retrieval of records from defunct systems or media

Keeping Digital Records (both in agencies and in the archives) Standards for ·  describing digital records ·  minimum requirements for preservable data formats ·  trustworthy digital repositories ·  maintenance of provenance and authenticity for digital records

Transferring Digital Records to Archives Guidelines on: ·  Preferred data formats & methods for transferring records to archival custody ·  Methods for automatic transfer of recordkeeping metadata ·  Maintenance of provenance and authenticity   Standards for ·  Transfer between government agencies and from agencies to archival custody ·  Minimum authenticity requirements ·  Transfer media

Using Digital Records Guidelines on: ·  Legal provisions re access to digital archives ·  Expunging sensitive data from public access copies ·  Storage & presentation of preserved data formats ·  Certification of provenance & authenticity ·  Fraud prevention ·  Data re-formatting and presentation (e.g. databases) ·  Searching & discovery mechanisms   Standards for ·   Uniform resource discovery based on metadata sets (e.g. AGLS) ·   Thesaurus-based searching

Uniform Australasian Approach Public standards ISO 14721:2003 (OAIS Blue Book) ISO15489 – Records Management XML rfc Managing the entire continuum Recordkeeping standards Metadata standards Genre schemas Implementation approaches

Uniform Australasian Approach Public standards ISO 14721:2003 - Reference Model for an Open Archival Information System

Uniform Australasian Approach Implementation-level approaches XML package wrapping Industry commodity platforms Open, documented formats Minimise migration requirements Provenance & authenticity mechanisms (e.g. checksums)

Uniform Australasian Approach What is not uniform? Legislative regimes Access regimes Implementation details State of play

ADRI: Projects/Products Common transfer/ingest format and/or standard for a Submission Information Package (Exposure Draft) Generic business cases for digital recordkeeping Generic specifications for digital archive hardware and software functionality Staff training workshops/modules Generic specs for digital archive hardware/software Generic specs for records management software Advice on Digital Rights Management Technology

Issues to be resolved? Whole of archives reinvention for end to end digital archiving Capacity planning/scalability Metadata encapsulated objects vs distributed metadata management? Digital Rights Management technology

ADRI Website http://www.adri.gov.au http://www.adri.govt.nz