Presentation on theme: "Shared Challenges, Gaps, and Needs"— Presentation transcript:
1 Shared Challenges, Gaps, and Needs Digital PreservationShared Challenges, Gaps, and Needs
2 Shared Challenges, Gaps, and Needs Reflections on the anniversary of the Report of the CLIR/RLG Task Force on Digital ArchivingThe grand challenges of Digital PreservationIntellectual property and the relation between preservation and accessA network of trusted institutions and the question of certificationThe interoperability gapBusiness models and models of cooperation
3 The Task Force on Archiving of Digital Information May 1st was the tenth anniversary of the reportFull disclosureThe test of time:Key findingsRecommendations
4 Key findings of the Task Force The first line of defense rests with the creators, providers and owners of digital information.Long-term preservation requires a deep infrastructure capable of supporting a distributed system of digital archives.A critical component of the digital archiving infrastructure is the existence of a sufficient number of trusted organizationsA process of certification for digital archives is needed to create an overall climate of trust about the prospects of preserving digital information.Certified digital archives must have the right and duty to exercise an aggressive rescue function as a fail-safe mechanism for preserving valuable digital information that is in jeopardy of destruction, neglect or abandonment by its current custodian.
5 Task Force Recommendations (1) Pilot projectsPreserve objects of the early digital ageIdentify and address economic and legal barriersApplications of technologies and services (emulation, IP transactions, object authentication mechanisms
6 Task Force Recommendations (2) Support structuresPreservation-friendly national policies (network pricing, security, tax incentives and accounting structures)Legal foundations for effective fail-safe archivesCommunity- and discipline-based archivesStandards, criteria, and mechanisms for certificationPoints of contact for international cooperation
7 Task Force Recommendations (3) Best practice case studiesFacilitating archiving at creationMassive storageMetadata standardsMigration paths
8 Preservation state of the art From prayers …James Gleick reported in The New York Times (“The Digital Attic: An Archive of Everything,” 12 April 1998) that "the Daiho Temple of Rinzai Zen Buddhism held a ‘memorial service for lost information’ in Kyoto and online."“After the effort of transforming all this knowledge into electronic information has been completed, is it enough then to say that we are finished? ... There are many 'living' documents and softwares that are thoughtlessly discarded or erased without even a second thought. It is this thoughtlessness that has drawn the concern and attention of Head Priest Shokyu Ishiko. Head Priest Ishiko hopes that through holding an "Information Service" and by teaching the words of Buddha, that this 'information void' will cease to exist” (http://www.thezen.or.jp/jomoh/kuyo.html).We need to help them!… To this workshop
9 The grand challenges of Digital Preservation Intellectual property and the relation between preservation and accessA network of trusted institutions and the question of certificationThe interoperability gapBusiness models and models of cooperation
10 Intellectual property Common complaint: it is very difficult to preserve intellectual property because copyright law limits copying for such purposes.The issue is especially complicated because of the popular formula— “preservation is access”This equation arose mainly to mobilize efforts to preserve out-of-copyright brittle books.But if preservation is a backdoor for redistribution of “at-risk” in-copyright materials then, rights-holders would resist
11 An aggressive rescue function The Task Force subcommittee addressing the IP topic had high aspirations for detailed recommendations, but generated little consensus and so dealt with intellectual property in a limited way.It focused on a simple call for the law, and particularly Section 108 of the copyright law, to allow for an aggressive rescue function.The Task Force formulation:“No distributed system of digital archives will afford effective protection of electronic information unless it provides for a powerful rescue function allowing one agency, acting in the long-term public interest of protecting the cultural record, to override another’s neglect of or active interest in abandoning or destroying parts of that record.”Neglect and willful abandonment and destruction are prominent attributes of the digital environment
12 Neglect and willful abandonment or destruction The loss of the Task Force correspondence archives and the preservation prayer.The judgments against Morgan Stanley for failing to preserve .Publishers appear to be highly vulnerable to legal demands, editorial second-guessing, and other activities that result in the removal of materials from their own archives.The Elsevier “Vanishing act” was documented on the LIBLICENSE listserv in 2004Such “acts” produce a “Swiss cheese” effect in the cultural recordJim O’Donnell: The “Vanishing Act” discussion “is disturbing, because it is the tip of the iceberg, I think: if for fairly transient reasons, publishers will pull articles, when might not publishers prove unreliable for other reasons?”
13 Solutions?Jane Ginsburg and June Besek are engaged in a Mellon-funded study of legal strategies for protecting archives that are preserving parts of the cultural record from being subject to takedown demandsEmergence of discussion of light versus dark archives, where “dark archives” restrict archivesWill restricted archives attract investment?The experience of LOCKSS and especially Portico is relevantThe Portico business model had to be changed to exclude deep investment in access, because publishers would not contribute and the access investment proved too expensive for librariesThe Section 108 Study Group recently took public comments on its version of the “aggressive rescue function:” a “preservation-only” exception to the copyright law
14 Preservation-only exception An exception would allow copying of “at risk” materials by qualified institutions from which access would be severely restrictedAccess would be articulated in the definition of “trigger conditions” allowing varying types of access depending on conditions such as:Monitoring by staff and “auditors”Need for use by qualified researchersCreating a “replacement copy,” if the work is no longer available on the market at a fair priceIf the work is abandoned or orphanedIf permission is givenWhen copyright expires
15 Conclusion on IP issues There are no easy answersAlthough preservation may not equal access, the “triggers” suggest that there is a relationship between preservation and access, and that it is subtle and nuanced.In the preservation arena, IP may need to be addressed by attention to nuance.
16 The grand challenges of Digital Preservation Intellectual property and the relation between preservation and accessA network of trusted institutions and the question of certificationThe interoperability gapBusiness models and models of cooperation
17 A network of trusted institutions The Task Force suggested that:“Repositories claiming to serve an archival function must be able to prove that they are who they say they are by meeting or exceeding the standards and criteria of an independently-administered program for archival certification.”Certification has attracted considerable attention.But trust is the central issue.
18 The Need for Trust Trust is important mainly in its absence: When there is a lack of confidenceWhen there are multiple interests in an object or activity and one or more parties lack controlWhen there is the potential for those in control to cause loss or harm, either deliberately or inadvertentlyIn the preservation context, there is a need for trust because:There is uncertainty about how to preserve information in a digital environmentThose interested in preserving the scientific and cultural record do not control itThe potential for loss is great and growing
19 Preservation features that build trust Technical ability:The repository must be able to maintain the authenticity and integrity of the informationThe information must demonstrably be what it purports to be.Organizational designA commitment to the preservation mission,Protections and controls against lossWell-defined preservation services
20 A significant body of work on trust in archives RLG/OCLC Working Group on Digital Archive Attributes (2002)RLG-NARA Task Force on Digital Repository Certification (2005)Center for Research Libraries is testing the feasibility of a certification process against a variety of existing or emerging archivesMeanwhile:The National Science Board report on “Long-Lived Data Collections” (2005) observed layers of responsibility (local, group, national) within scientific communities with the need for oversight growing as the collections are more important to each communityThe Digital Preservation Coalition study, Mind the gap: Assessing digital preservation needs in the UK (2006) has generalized the point to suggest that different communities have different needs for regulation and certification of preservation activity.
21 Enhancing trust through the law? Preservation repositories certainly need to be protected against “takedown” ordersSpecific, highly regulated industries might require preservation certification as a condition of doing businessIn general, a consensus is emerging thatcertification should be community-driven rather than mandated centrally for all repositories because it would be too difficult and cumbersome to administer as a unified process across communities with differing requirements and needsOpen questions: Where are the communities of common interest? How do they encourage openness and transparency in the interest of establishing trust? What standards need to be enforced? How general are these standards across communities? When is it in the self-interest of preservation agents to submit to certification and audit processes?
22 Open questions Where are the communities of common interest? How do they encourage openness and transparency in the interest of establishing trust?What standards need to be enforced?How general are these standards across communities?When is it in the self-interest of preservation agents to submit to certification and audit processes?
23 The grand challenges of Digital Preservation Intellectual property and the relation between preservation and accessA network of trusted institutions and the question of certificationThe interoperability gapBusiness models and models of cooperation
24 The interoperability gap There is room for extensive cooperation among archives, including various kinds of divisions of labor for more efficient operationsAn example: preservation requires bulk transfer of content across and among repositoriesWhat standards and services are needed?PREMIS data model and data dictionaryFormat registriesNeed for practical experiments and testingNDIIPP Archive Ingest and Handling TestPreliminary results of the New York workshop on “Augmenting Interoperability across Scholarly Repositories”
25 Augmenting Interoperability Focus on complex digital objectsKey service functions need to be represented in repository interfacesGet/Obtain/HarvestPut, or request for submissionNotion of “surrogate” as a representation of the complex object: Does it reference the object or contain it?Generalized data model:Unique identifier (need standard ways to locate repository)Other features: lineage or provenanceNeed for substantial inter-repository experiments in multiple domains (e.g. chemistry; museums; archaeology; preservation) to formulate and test the general data model.
26 The grand challenges of Digital Preservation Intellectual property and the relation between preservation and accessA network of trusted institutions and the question of certificationThe interoperability gapBusiness models and models of cooperation
27 Business modelsPreservation depends on commitment AND sustained resourcesHow to generate sustained resources?Preservation is a public good and is subject to free-riding problemsGovernment tax and funding addresses the problem, but is it enough?Libraries, archives, and museums may be able to take responsibility as part of their mission, but economies of scale is an issueService bureaus solutions work at scale but may be operating in special classes of enterprise such as those in a two-sided market
28 Two-sided markets Credit cards are a classic case How to crack the chicken-egg problem: customers won’t use the card if stores don’t accept it; stores won’t accept the card if there aren’t enough customersThe Visa solution: charge the merchant; the AMEX “prestige” solution: charge bothLOCKSS and Portico both have struggled with this problem of publisher and library participationPortico required several rounds of intensive negotiations and marketing analyses on both sidesAbandoned initial interest in access provisions and lowered costs in what might be characterized as an insurance modelAre other business models appropriate in other arenas?
29 Resources depend on interaction with commercial entities Commercial versus public and not-for-profit solutions?Commercial interests in preservation are powerful if the IP is profitableGovernment funding depends on competition in a political processNot-for-profit missions may be divided; in libraries, for example between preservation and accessThere need to be imaginative approaches to developing shared vision and mutual support across sectorsLearn and draw from commercial preservation expertise in film, sound recording, pharmaceutical, and other industriesCommercial uses of preservation archives: indexing; content from one publisher to another; addressing perpetual access claimsMore imagination needed?
30 Next steps The grand challenges of Digital Preservation Intellectual property and the relation between preservation and accessA network of trusted institutions and the question of certificationThe interoperability gapBusiness models and models of cooperationThese are some suggested areas of priority