1. Bringing Self-Assessment Home: 1 Bringing Self Assessment Home: Repository Profiling & Key Lines of Enquiry Within DRAMBORA Andrew McHugh, Perla Innocenti, Seamus Ross, Raivo Ruusalepp, Hans Hofman Digital Curation Centre (DCC), DigitalPreservationEurope (DPE), HATII at the University of Glasgow & National Archives of the Netherlands

2. Bringing Self-Assessment Home: 2 Trustworthiness and Archival Stewardship responsibility Trustworthiness is an increasingly sought after commodity Decentralization part of a normal progression (see UK AHDS)‏ Trustworthiness has wide reaching implications –external (financiers, depositors, creators, consumers)‏ –internal (management, strategic planning)‏

3. Bringing Self-Assessment Home: 3 Attempts to meet this demand Several mainstream reference materials available –Trustworthy Repositories Audit & Certification Criteria and Checklist (TRAC)‏ –nestor's Catalogue of Criteria for Trusted Digital Repositories Audit check-lists designed to support conferment of certification Examples of top down assessment philosophy

4. Bringing Self-Assessment Home: 4 Top down approach is tried and tested Many auditable domains benefit from objective criteria –Information and IT security –Financial regulation But disregards diversity evident across preservation discipline –funding, scale, legislative responsibilities and restrictions, content types, technology and policy vary Generic criteria are difficult to conceive

5. Bringing Self-Assessment Home: 5 Something is missing Check-lists are potentially invaluable But criticisms continue of non-applicability (or at least not total applicability)‏ A more tailorable assessment solution capable of acknowledging atypical repository characteristics is required The Digital Repository Audit Method Based on Risk Assessment is our response

6. Bringing Self-Assessment Home: 6 Attempting to define “the repository” An intellectual context for the work: –Commitment to digital object maintenance –Organisational fitness –Legal & regulatory legitimacy –Effective & efficient policies –Acquisition & ingest criteria –Integrity, authenticity & usability –Provenance –Dissemination –Preservation planning & action –Adequate technical infrastructure (CRL/OCLC/NESTOR/DCC/DPE meeting, January 2007)‏ © HATII UofGlasgow, 2007

7. Bringing Self-Assessment Home: 7 The Perils of Objectivism Difficulties associated with a generalization of optimal repository characteristics Do all repositories really share singularity of purpose? Uniform priorities? Documenting a set of blue sky aspirational repository qualities is useful – nestor and TRAC are compelling reference materials But both check-lists are necessary vague

8. Bringing Self-Assessment Home: 8 Audit and Certification: Two fundamental Challenges Although inevitably related, these two challenges are distinct in terms of their infrastructural dependencies Certification is well served by objective check-lists Comparability of results is necessary to enable meaningful assertion of performance in a wider context

9. Bringing Self-Assessment Home: 9 The Audit as discrete process Undoubtedly an essential precursor to the award of certification Best practice guidelines/check-lists provide an intellectual foundation for audit But... neither TRAC nor nestor provide (explicitly or otherwise) a tangible structure for determining where conformity is evident – the mechanics of audit

10. Bringing Self-Assessment Home: 10 The Evolution of an Audit Methodology Digital Curation Centre (DCC), DigitalPreservationEurope (DPE), and the DELOS project have been undertaking pilot audits for almost two years In March 2007 an audit methodology was released to reflect the findings Subsequent pilots have aimed to validate the methodology's effectiveness - audit as a standalone process has demonstrable value

11. Bringing Self-Assessment Home: 11 Digital Repository Audit Method Based on Risk Assessment Two discrete phases of self-assessment, reflecting the realities of audit; –Information accumulation, aggregation and documentation –Risk identification, assessment and response Preservation is fundamentally a risk management process –Risks are implicit in every aspect of repository activities –Borne or influenced by numerous contextual factors

12. Bringing Self-Assessment Home: 12 What good is self assessment? Comparability of disparate repositories? Such questions misunderstand the complex realities of the repository landscape “Repository” means very little Which of the following can't legitimately be described as a repository? –Website, database, CRM system, banking system, eLearning or eResearch environment, digital library, blog, wiki, personal desktop...

13. Bringing Self-Assessment Home: 13 Comparability of unrelated repositories is a non-issue Even assuming smaller subset of 'preservation repositories' Repositories won't be selected or financed based only on results of certification Services are critical, with performance understood in terms of those services We want to identify and describe classes of repositories in terms of their common services and characteristics

14. Bringing Self-Assessment Home: 14 Repository Classification Self assessors can space their own efforts within the context of comparable repositories They can reflect and inform the perspective of best practice that exists within their own particular 'repo-sphere'. Maybe no two repositories are identical, but each characteristic will likely be shared elsewhere

15. Bringing Self-Assessment Home: 15 Perils of subjectivism DRAMBORA is fundamentally 'bottom-up' Comparability and reproducibility of results are compromised More concerning is that improvement in self assessment is limited by one's own horizons How can repositories comment on unanticipated risks? When they are unaware of available opportunities?

16. Bringing Self-Assessment Home: 16 Finding Islands of Objectivity 80 or so example risks to prompt thinking... insufficient DRAMBORA Interactive enables repositories to align their objectives, activities, strengths and shortcomings with other peer repositories' responses This will ultimately be collated as a series of repository profiles encapsulating key roles, responsibilities, functions and risks

17. Bringing Self-Assessment Home: 17 DRAMBORA Interactive www.repositoryaudit.eu

18. Bringing Self-Assessment Home: 18 Key Lines of Enquiry DRAMBORA's authors are looking at the development of 'key lines of enquiry' Detailed question sets intended to inform and regularize the assessment process. Risk → Practical Response → Will and Capacity? → Example Vulnerabilities

19. Bringing Self-Assessment Home: 19 Example Risk: Identifier to information referential integrity is compromised -it becomes impossible to associate identifiers and information. TRAC Criterion: If unique identifiers are associated with SIPs before ingest, the repository preserves the identifiers in a way that maintains a persistent association with the resultant archived object (e.g., AIP).

20. Bringing Self-Assessment Home: 20 Example (2)‏ Risk Responses: –Objects are renamed to correspond with identifiers –Objects are stored in a directory named to correspond to identifier –Objects are packaged using alternative mechanism with identifier information (e.g., in a zip file with associated text file)‏ –Database table maintains identifier with corresponding field describing full path

21. Bringing Self-Assessment Home: 21 Example (3)‏ Key Lines of Enquiry: –Does repository apply its own identifiers or maintain existing ones for information packages? –Under what circumstances could identifier collision occur? –Is a bespoke or off-the-shelf (e.g. Handle, DOI, PURL) identifier scheme employed? –Are third party resolver services required? –What overt costs are associated? –In what circumstances could the identifier become divorced from the related object? –What redundancy is employed to maintain referential integrity?

22. Bringing Self-Assessment Home: 22 Example (4)‏ Vulnerabilities or Consequences: –Maintained identifiers are duplicated –Time stamp collisions –Recorded paths are obsolete

23. Bringing Self-Assessment Home: 23 Conclusion DRAMBORA Interactive has to offer more than just increased usability A role of audit facilitator, injected with sufficient scope and functionality to lead auditor through the process In isolation, or combined with objective guidelines, DRAMBORA offers benefits to repositories both individually and collectively