Hydra, research data and Archivematica 6/9/2018 Hydra, research data and Archivematica Julie Allinson (York) and Richard Green (Hull) Hydra Connect 2016– Boston, MA – 5th October 2016

Research data management Why do we need digital preservation for research data? Often unique and irreplaceable May be needed to validate conclusions reached in publications May have a high-level mandate to do so May have potential for re-use Doesn’t our repository do that? Many (most?) repositories are designed for medium-term curation and access rather than long-term preservation Preservation implies actively taking steps to increase the chances of enabling meaningful re-use in the future

Enter Archivematica Archivematica is “a web- and standards-based, open-source application which allows your institution to preserve long- term access to trustworthy, authentic and reliable digital content.” “…in compliance with the ISO-OAIS functional model.” York and Hull both keen to see how Archivematica might potentially fit into our preservation workflows

Jisc “Research Data Spring” funding Late 2014 Jisc made grant funding available for universities to investigate ways of managing research data (to satisfy a Government mandate) Hull and York jointly awarded money to look at the role Archivematica might play in an RDM workflow: “Filling the Digital Preservation Gap”

Jisc “Research Data Spring” funding Led to three phase project 2015-2016 (each phase bid for separately!): Phase 1 : Desk research Phase 2 : Work with Archivematica and PRONOM teams to make Archivematica a better fit to the need – the project actively funded development work - and to develop local implementation plans for integrated systems Phase 3 : Further improvements (particularly to file format identification via DROID) and build proof-of-concept (p-o-c) systems at Hull and York

Jisc Research Data Shared Service Almost in parallel with the Research Data Spring projects Jisc were planning a Research Data Shared Service The resulting system will be managed and hosted, and will offer three core modules : repository, preservation and reporting Phase 1 and 2 reports from Hull and York very influential for the preservation module Commercial and open source offerings for each module, including Archivematica (for preservation) and Hydra (for the repo) Over 20 pilot institutions recruited (including York) – all identified preservation as a priority

York p-o-c implementation York wanted to provide: an easy way of depositing data a way of monitoring datasets for RDM staff a way of requesting access to data with: data sent to archivematica dataset metadata pulled from PURE

York p-o-c implementation Metadata from PURE pulled in nightly or on-demand Visual representation of workflow status Fedora objects created for the dataset to store local admin info and help connect the PURE and Archivematica records

York PCDM modelling Dataset = Dataset record from PURE Dataset can be made up of multiple ‘Packages’ of data, eg. newer version Individual data files stored, but folder structure is not Folder structure available in Archivematica METS

York p-o-c outputs Code: https://github.com/digital-york/researchdatayork https://github.com/digital-york/dlibhydra Data model (draft): https://docs.google.com/document/d/1QPw9kLqRFz I5aStRr3nlBqqj5BzkI3eFZY4zNjOmo8w/edit?usp=sha ring

What next in York? Our RDM staff love the p-o-c and we have agreement to turn it into a production system over the autumn/winter This has been a helpful exercise for broader data modelling / Hydra implementation at York

Hull p-o-c implementation Hull keen to make Archivematica part of a workflow for any type of repository content – not just research data. You may have seen a poster at Hydra Connect last year: Hull’s p-o-c implements most of the automated bulk ingest route, creates AIP(s) and builds repository objects from the DIP(s)

Hull p-o-c implementation User assembles files and simple descriptive file(s) in Box folder. Shares the folder with Archivematica System checks folder contents and if OK creates a bag (BagIt standard) for each object which is passed to Archivematica Archivematica processes the bag to create an AIP which goes to a preservation store… …and also a DIP which is passed to the DIP processor DIP processor creates Hydra objects from the DIP contents and injects them into the repository QA queue… …matched to the AIP by UUID Thanks to Cottage Labs for all the new development work!

Hull p-o-c options Depositors have several options: A folder containing multiple data files and one descriptive file  a single AIP and a single repository object with (optionally) one or more surrogate files for download (so can be a “metadata-only” record) A folder containing multiple files and a csv file (one row per file)  multiple AIPs with multiple repository objects, each with (optionally) a surrogate for download A folder containing the top-level folder of a structure  a zipped structure in a single AIP and a single repository object (optionally) containing the zipped file for download

What’s next in Hull? We hope to be able to take the p-o-c work and turn it into a production system Hull is the UK’s “City of Culture” next year and there will be a great deal of digital material that the University Archives want to capture for posterity

Digital Preservation Awards 2016 We’re flattered that the project has been nominated as one of the three finalists for the Digital Preservation Awards “Research and Innovation Category” this year

Project reports The project reports for “Filling the Digital Preservation Gap” are available through Hull’s repository hydra.hull.ac.uk - search for the project title The phase one and two reports are there now, phase three by the end of the month

Thanks! Questions? julie.allinson@york.ac.uk r.green@hull.ac.uk Hull github: https://github.com/orgs/uohull Colleagues Chris Awre (Interim Librarian, University of Hull) c.awre@hull.ac.uk Jenny Mitcham (Digital Archivist, University of York) jenny.mitcham@york.ac.uk Simon Wilson (University Archivist, University of Hull) s.wilson@hull.ac.uk Archivematica - www.archivematica.org