Digital Preservation MetaArchive Cooperative, Digital Preservation Policy Planning Workshop Boston College, Boston, MA October 26, 2010

MetaArchive Cooperative, Digital Preservation Policy Planning Workshop Boston College, Boston, MA October 26, 2010

 MetaArchive Cooperative  Preservation Committee ▪ Bill Donovan ▪ Rachel Howard ▪ Susan Parham ▪ Mark Stoffan ▪ Katherine Skinner ▪ Matt Schultz Schultz, Donovan, Howard, Skinner, 2010

 Digital Preservation Policy Building  University Libraries have begun the process of engaging digital preservation ▪ Performing digitization ▪ Building digital libraries ▪ Establishing institutional repositories  Digital Preservation Policies have often lagged behind digital preservation activities ▪ The result is a myriad non-standard developments Schultz, Donovan, Howard, Skinner, 2010

 Policies can also drive development and standardization  MetaArchive Preservation Committee  Began reviewing successful policies ▪ Found a lot of commonalities  Developed a Digital Preservation Policy Template  Developed a Digital Assets Survey Schultz, Donovan, Howard, Skinner, 2010

 Session 1: Digital Preservation Trends  Session 2: Preservation Policy Overview  Resource: Policy Template  Break  Session 3: Planning for Policy Development  Resource: Digital Assets Survey  Session 4: Policy Development Schultz, Donovan, Howard, Skinner, 2010

 Not a Final Product!  We need your feedback!  We will have Q&A breaks and activities  But feel free to ask questions anytime  We are taking a top-down approach to digital preservation – BIG POLICY DEVELOPMENT  Not all institutions are the same  Feel free to tailor our resources to any scale Schultz, Donovan, Howard, Skinner, 2010

Digital Preservation MetaArchive Cooperative, Digital Preservation Policy Planning Workshop Boston College, Boston, MA October 26, 2010

Session 1

 What is Digital Preservation?  Trends in Digital Preservation The Goal: To understand the coalescing landscape of digital preservation requirements and consider the potential investments needed for developing a policy driven approach to digital preservation. Schultz, Donovan, Howard, Skinner, 2010

 “The series of managed activities necessary to ensure continued access to digital materials for as long as necessary.” - Definition from Digital Preservation Coalition Schultz, Donovan, Howard, Skinner, 2010

 Centralized & Distributed Preservation  Full & Bit-level Preservation  Preservation Metadata  Open Source solutions  Focus on economies of scale and benefits  Roles & Responsibilities  Sustainability  Standards and auditing metrics  National mandates  Avoiding silos & pursuing interoperability Schultz, Donovan, Howard, Skinner, 2010

 Centralized preservation:  Preservation activities managed by single institution  Examples: ▪ Chronicling America ▪ DAITSS  Distributed preservation:  Preservation activities managed by multiple institutions replicating and/or geographically locating collections  Examples ▪ LOCKSS ▪ MetaArchive Cooperative ▪ Chronopolis Schultz, Donovan, Howard, Skinner, 2010

 Many archives doing a bit of both  Something of a false dichotomy  Full Preservation  Focuses heavily on format migration and normalization (may still preserve the original) ▪ Highly concerned with monitoring and intervening against format obsolescence up-front  Bit-level Preservation  Focuses primarily on preserving the original bits ▪ Avoids migration, normalization, and monitoring up-front and cites long-lived support or convertability of the majority of formats Schultz, Donovan, Howard, Skinner, 2010

 PREMIS  Administrative metadata  Technical metadata  Structural metadata  Provenance metadata  Metadata standards are always under development – mark the moment to learn and continue to watch the horizon Schultz, Donovan, Howard, Skinner, 2010

 Open source is a well recognized best practice at this point – real question is: How open?  Why Open Source?  Avoiding proprietary solutions can guard against dependencies and sudden loss  Open source formats and technologies maximize communities of support and ensure flexibility and long-lived solutions  Open source approaches dramatically reduce technology costs and can lead to building of expertise Schultz, Donovan, Howard, Skinner, 2010

 Digital preservation needs are great at most institutions and digital preservation can be costly  You don’t have (shouldn’t try) to save everything!  Blue Ribbon Task Force on Sustainability  Economies of scale can reduce staff costs  Focus on communicating the benefits to the institution aids in selection and prioritization ▪ Prioritization crucial for offsetting costs ▪ Define the institutional value of your assets Schultz, Donovan, Howard, Skinner, 2010

 Partnering with other institutions to preserve content is becoming more popular  Sharing resources and expertise reduces costs  Maintains control over institutional assets rather than handing over responsibility to third parties  Consumers also becoming Producers and Preservers of digital assets  Modularizing the chain of preservation activities (ingest, storage, dissemination)  Microservices and interoperability Schultz, Donovan, Howard, Skinner, 2010

 Many grant funded projects are short-lived or narrowly focused  Institutions have been pressured to just enter the game and hope for the best  Diverse revenue streams becoming essential  NDIIPP transitions to NDSA  Emphasis on Collaboration  Promoting self-sustaining cost models Schultz, Donovan, Howard, Skinner, 2010

 Trustworthy digital repositories!  Reference Model for an Open Archival Information System (OAIS)  Trusted Repositories Audit & Certification (TRAC) – 2007  Metrics for Digital Repository Audit & Certification – awaiting ISO standardization Schultz, Donovan, Howard, Skinner, 2010

 Scientists seeking funding will soon be required to submit data management plans – NSF Press Release (May 10, 2010)  Ensuring long-term accessibility and sharing of data and digital assets to improve research  There is no access without preservation  A massive undertaking requiring top-down institution-wide policies Schultz, Donovan, Howard, Skinner, 2010

 Information, data, and research silos result from institutional management structures  Result is a multiplicity of practices and technologies that prevent sharing and re-use  An acknowledged problem  We’re just getting started on solutions  Institution-wide policies have potential to help catalyze institutional change and break down silos Schultz, Donovan, Howard, Skinner, 2010

Digital Preservation MetaArchive Cooperative, Digital Preservation Policy Planning Workshop Boston College, Boston, MA October 26, 2010

Session 2

 Libraries as Ideal Curators  Policies as Catalytic Solutions  What is a Digital Preservation Policy?  Building on Successful Digital Preservation Policies  Preservation Policy Template Schultz, Donovan, Howard, Skinner, 2010

The Goal: To get a glimpse into the higher level concerns that a digital preservation policy attempts to address and the statements used to reflect your Library’s strategic positions. Schultz, Donovan, Howard, Skinner, 2010

 Libraries as Curators for the Public Good  Only librarians understand and appreciate the needs of their collections and their users  Irreplaceable stewards of our collective cultural memory  Avoiding the Broker trap  Constantly outsourcing preservation roles, services & infrastructure may threaten institutional mission and societal role  Reversing the Trends  Private LOCKSS Networks are enabling libraries to maintain control of research data and digital collections  IIPC is enabling libraries to preserve our shared Web culture  Can you think of some other examples? Schultz, Donovan, Howard, Skinner, 2010

 Success of the MetaArchive Cooperative  Reducing the cost for libraries to engage preservation of their collections through shared resources and open source technology  Empowering libraries and other cultural memory institutions through growing expertise and embedding infrastructure in the libraries  Actively Addressing Trends in Digital Preservation Schultz, Donovan, Howard, Skinner, 2010

 Policies can trigger collaboration and action!  Impacts of Institutional Policy Building  Content Policy (MetaArchive Cooperative) ▪ Solidified shared commitments while retaining institutional flexibility  ETD Preservation Policy (Boston College) ▪ Defined institutional commitment and responsibility and achieved administrative buy-in  Other institutional policy examples ▪ Promoting shared infrastructure (ISU) ▪ Guaranteeing open access (FSU) ▪ Pursuing reliable research (Georgia Tech) Schultz, Donovan, Howard, Skinner, 2010

 “Digital preservation policies document an organization’s commitment to preserve digital content for future use; specify file formats to be preserved and the level of preservation to be provided; and ensure compliance with standards and best practices for responsible stewardship of digital information.”  From Long Definition of Digital Preservation, prepared by the ALCTS Preservation and Reformatting Section, Working Group on Defining Digital Preservation, accessed at /defdigpres0408.cfm. /defdigpres0408.cfm Schultz, Donovan, Howard, Skinner, 2010

 Digital Preservation Policy Template  General Questions for Feedback ▪ Are there additional policy sections that would be helpful for your institution to define? ▪ What are the policy areas that would require the most effort at your institution to define? ▪ How and where would such a policy be promoted and publicized? ▪ Who are the most important audiences for a policy of this type? Schultz, Donovan, Howard, Skinner, 2010

 This is a simple statement that relates digital preservation to the institution’s mission and the communities it serves. Schultz, Donovan, Howard, Skinner, 2010

 This is a set of simple paragraphs that summarize the overall intent of the institution. Schultz, Donovan, Howard, Skinner, 2010

 This section summarizes the resource groups (e.g., units, departments, or external parties) for which the institution takes responsibility and prioritizes these according to institutional importance. Schultz, Donovan, Howard, Skinner, 2010

 This section outlines the way decisions are made regarding what will be preserved. Schultz, Donovan, Howard, Skinner, 2010

 This section summarizes the lifecycle management practices of the institution. Broad categories might include content creation, content integrity, and content maintenance. Schultz, Donovan, Howard, Skinner, 2010

 This section provides overview of methodologies and philosophies undergirding preservation activity (e.g., OAIS, TRAC, etc). Schultz, Donovan, Howard, Skinner, 2010

 This section details who is involved and at what level they are involved. Who is charged with preservation responsibility? Schultz, Donovan, Howard, Skinner, 2010

 This section describes policy/policies for ascribing metadata to preservation objects. May include schema references or documentation bodies (e.g., LC, DLF, NARA). Schultz, Donovan, Howard, Skinner, 2010

 This section documents policies around permissions and access of preserved content. Schultz, Donovan, Howard, Skinner, 2010

 This section contains information about what the institution’s relationship is to other institutions, and whether it may partner with other institutions to preserve its own collections or the collections of other institutions and under what circumstances. Schultz, Donovan, Howard, Skinner, 2010

 This section documents expected costs and who shoulders the responsibility for those costs. Schultz, Donovan, Howard, Skinner, 2010

 This section acknowledges the challenges the institution/field faces in preserving digital collections. Schultz, Donovan, Howard, Skinner, 2010

 This section gives an overview of any outreach and education activities undertaken by the institution. Schultz, Donovan, Howard, Skinner, 2010

 This section provides the date of last revision and contact information for the authors Schultz, Donovan, Howard, Skinner, 2010

 This section lists other institutional documentation that has a relationship to digital preservation and/or this policy itself. Examples might include such documentation as Disaster Plan, Records Management Policy, and Collections Development Policy. Schultz, Donovan, Howard, Skinner, 2010

 This section would clarify terms used throughout the document. Schultz, Donovan, Howard, Skinner, 2010

Digital Preservation MetaArchive Cooperative, Digital Preservation Policy Planning Workshop Boston College, Boston, MA October 26, 2010

Session 3

 Libraries Leading the Way  Policy Building Costs & Benefits  Policy Preparation Schultz, Donovan, Howard, Skinner, 2010

The Goal: To familiarize policy developers with the assessment and development activities that may need to take place to bridge policy with reality. Schultz, Donovan, Howard, Skinner, 2010

 First Steps  Get the green light from upper library administration to form an exploratory team ▪ Who would lead this charge at your institution? ▪ What does that process look like? ▪ How are such teams formed? ▪ Anticipate the politics? Schultz, Donovan, Howard, Skinner, 2010

 Establish an exploratory committee of:  Library Heads  Digital Librarians and Archivists  Content/Collection Specialists  Catalog Specialists…  …to research and define:  Policy Building Costs & Benefits  Policy Preparation Schultz, Donovan, Howard, Skinner, 2010

 Define the costs of not preserving your institution’s digital assets  Operational Costs?  Scholarly & Scientific Research?  Institutional Reputation?  Define the potential benefits of building an institutional policy  Integrated Workflows and Cost Reductions  On-going Scholarly Access and Use (re-use)  Digital Expertise and Leadership in the Field Schultz, Donovan, Howard, Skinner, 2010

 Define the costs of building a policy  Dedicating staff hours  Building inter-departmental cooperation  New administrative, departmental, and library staff training and procedures  New technology & infrastructure investments  Exploring legal obligations (IP, Partner Institutions, etc.)  Other? Schultz, Donovan, Howard, Skinner, 2010

 Preparing to build a Digital Preservation Policy for your institution requires a thorough assessment of where you are and where you should be – let’s take a look! Schultz, Donovan, Howard, Skinner, 2010

 We’ll get to these later:  Policy Statement  Summary Statement  Let’s start with:  Scope & Selection Criteria Schultz, Donovan, Howard, Skinner, 2010

 Defining digital assets at your institution  Digitized (Ex: scanned newspapers)  Born-digital (Ex: websites)  Electronic records (Ex: spreadsheets, databases, s)  Digital Research Data (Ex: raw sensor data)  Where do your digital assets reside?  At the departmental & unit level? Outside your institution?  Who are the major producers and consumers?  Researchers? Scholars? External parties?  Can they be deposited for preservation?  To what extent? Schultz, Donovan, Howard, Skinner, 2010

 Digital Assets Survey  Points for Feedback ▪ What additional questions might you ask? ▪ What questions seem extraneous or problematic? ▪ What form should this take for your institution? Paper survey? Electronic? ▪ How would results be gathered? ▪ How would you follow-up with respondents? Schultz, Donovan, Howard, Skinner, 2010

 Summarize the resource groups (e.g., units, departments, or external parties).  Who are the departments and individuals you might need to coordinate with to facilitate a successful survey distribution to all the potential resource groups that might exist across your institution. Schultz, Donovan, Howard, Skinner, 2010

 Outlines the way decisions are made regarding what will be preserved.  Based on survey feedback and follow-up interviews with resource groups, can begin to grasp the range of digital assets, their preservation needs, and how they should be prioritized for selection. Schultz, Donovan, Howard, Skinner, 2010

 Summarize the lifecycle management practices of the institution.  Perform a technical assessment of your Library’s existing approaches and capacity for creating, and maintaining digital assets.  Factor in a organizational structure, staffing and skill sets.  Address issues of quality control Schultz, Donovan, Howard, Skinner, 2010

 Technical Assessment  Library IT and Campus IT - Relationships  Hardware & Software Policies and Purchasing  Storage & Storage Management Environment(s)  Existing Back-Up Measures or Archiving Practices  Inventory and File Management  Quality Measures and Replacement Cycles Schultz, Donovan, Howard, Skinner, 2010

 Organizational Structure/Staffing  Identify duties required to support digital preservation  Identify staff with adequate skills and expertise to fulfill those duties  Review staffing plans, position descriptions, develop a matrix of duties and staff skills and expertise  Implement professional development training Schultz, Donovan, Howard, Skinner, 2010

 Quality Control  Identifying acceptable files & formats  Defining preservation levels and migration policies  Building a Preservation Plan  Enforcing risk assessment and technical evaluation schedules Schultz, Donovan, Howard, Skinner, 2010

 Provide an overview of methodologies and philosophies undergirding preservation activity  Communicate position toward trustworthy preservation by identifying steps taken to ensure use of standards (OAIS), transparency (willingness to engage in audit and review - TRAC) and accountability (making documentation available). Schultz, Donovan, Howard, Skinner, 2010

 Detail who is involved and at what level they are involved. Who is charged with preservation responsibility?  Preservation responsibility will undoubtedly be a joint endeavor (particularly between your Library, campus IT, and other external parties) and policy should reflect solidified agreements between all parties charged with responsibility. Schultz, Donovan, Howard, Skinner, 2010

 Central Administration  Evaluators? Enforcers? Consultants?  Institutional Departments & Units  Producers? Evaluators? Curators? Consultants?  Librarians & Archivists  Preservation Services? Curators?  Designated Community  Consumers and Users?  External Partners  Producers? Consumers? Preservation Services? Evaluators?  Vendors  Preservation Services? Consultants? Schultz, Donovan, Howard, Skinner, 2010

 Describe policy/policies for ascribing metadata to preservation objects.  Metadata is increasingly becoming central to trustworthy preservation, and statements of policy should be able to articulate your Library’s position on responsibility for capturing some level of preservation metadata, and the role it will play in managing that metadata on behalf of the digital objects you collect. Schultz, Donovan, Howard, Skinner, 2010

 Document policies around permissions and access of preserved content.  Consult with legal representatives for your institution  Research rights issues related to digital preservation  Limit your liability and develop a strategy for breach of copyright and removal of offending content Schultz, Donovan, Howard, Skinner, 2010

 Considerations:  The right to change or alter digital information is often limited by law to the creator  Digital program contracts must address the need to be able to work with and potentially modify digital objects to keep them accessible.  Agreements with depositors must specify and/or transfer rights to the program enabling appropriate and necessary preservation actions for the digital object.  Third party organizations should guarantee that relevant contracts, licenses, etc express rights, responsibilities and expectations of each party. Schultz, Donovan, Howard, Skinner, 2010

 Establishing the options for access and use of your institutions’ digital assets will go a long way toward both defining what sorts of management and dissemination workflows might need to be developed, as well as how to communicate the terms of such access and use. Schultz, Donovan, Howard, Skinner, 2010

 Define your institution’s relationship to other institutions, and whether it may partner to accomplish preservation.  Rights & Agreements ▪ Navigating the rights issues related to the digital objects that fall under your Library or institution’s preservation responsibility will go a long way toward articulating the terms under which partnerships can be pursued to further preservation development and activities. Schultz, Donovan, Howard, Skinner, 2010

 Document expected costs and who shoulders the responsibility for those costs.  Policies should confidently communicate an acknowledgement of the types of preservation activities that are in need of on-going financial support and general strategies that the Library will pursue to ensure that these activities will be adequately supported. Policy statements should be under-girded by responsible financial planning, accounting, and management. Schultz, Donovan, Howard, Skinner, 2010

 Financial Sustainability:  Sustainable Management & Financial Plans  Multi-Year Budget ▪ Factoring in financial cycles  Review Schedules (annual)  Seek diverse revenue streams to support preservation activities Schultz, Donovan, Howard, Skinner, 2010

 Acknowledge the challenges the institution/field faces in preserving digital collections.  Remember Trends in Digital Preservation  Perform Risk Assessment Schultz, Donovan, Howard, Skinner, 2010

 Trends in Digital Preservation  Centralized & Distributed Preservation  Full & Bit-level Preservation  Preservation Metadata  Open Source solutions  Focus on economies of scale and benefits  Roles & Responsibilities  Sustainability  Standards and auditing metrics  National mandates  Avoiding silos & pursuing interoperability Schultz, Donovan, Howard, Skinner, 2010

 Risk Assessment  Committing to analyze and report on risk, benefit, investment and expenditures  Identifying the real and potential threats to the digital preservation program, the digital collections, producers and consumers  Should include an inventory of file formats, technology infrastructure, legal mandates, staffing, etc. Schultz, Donovan, Howard, Skinner, 2010

 Give an overview of any outreach and education activities undertaken by the institution.  Champion your policies  Share your development  Develop workshops  Join coalitions and working groups  Know your sphere of influence  Be open to learning! Schultz, Donovan, Howard, Skinner, 2010

 Policy Statement  Summary Statement Schultz, Donovan, Howard, Skinner, 2010

 A simple statement that relates digital preservation to the institution’s mission and the communities it serves.  Review your institutions broader mission statements  Consider other legal, ethical, and policy mandates  Articulate the needs and the opportunities related to your institution’s resource groups Schultz, Donovan, Howard, Skinner, 2010

 A set of simple paragraphs that summarizes the overall intent of the institution.  Why does it preserve content (e.g., institutional, legal, consortial obligations)?  Who wrote this policy?  How often is this policy re-evaluated and by whom? Schultz, Donovan, Howard, Skinner, 2010

 Date/Author  Related Documents  Definitions/Glossary Schultz, Donovan, Howard, Skinner, 2010

Digital Preservation MetaArchive Cooperative, Digital Preservation Policy Planning Workshop Boston College, Boston, MA October 26, 2010

Session 4

 Moving the Committee Forward  Policy Development Process  Policy Development Timeline  Getting Policy Approval Schultz, Donovan, Howard, Skinner, 2010

 Secure the input of reliable policy enablers and force multipliers (Faculty? Staff? Other?)  Use the survey to generate interest and support ▪ Identify most interested departmental or unit stakeholders ▪ Identify faculty in key positions of influence ▪ Don’t forget campus IT!  Who are these people at your institution?  How would you get them involved? Schultz, Donovan, Howard, Skinner, 2010

 Discuss your policy preparedness  Discuss your institution’s identified digital assets  Digital Assets Survey  Draft an institutional digital preservation policy  Digital Preservation Policy Template  Develop a roadmap to implementation  Account for under-resourced policy areas Schultz, Donovan, Howard, Skinner, 2010

 Process will require substantial faculty commitment and participation so timing is critical  Avoid the beginning and end of semesters and the summer as so many faculty are preoccupied or away  At least a year or more might be required to gain support for a preservation policy. Schultz, Donovan, Howard, Skinner, 2010

 Establish regular reporting to Deans and Provosts on progress toward benchmarks  Face-to-face meetings  What does upper administration need to know to sustain and ultimately give approval?  Point to institutional Mission and Vision  Point to cost savings & benefits ▪ Tie to operational costs, research needs & reputation  Address digital preservation needs & trends  Suggest a viable roadmap to implementation  How might this need to be packaged to be concise and effective for your institutions? Schultz, Donovan, Howard, Skinner, 2010

 Digital Preservation Policy Template  Digital Assets Survey Schultz, Donovan, Howard, Skinner, 2010

 Will populate after final feedback from Pres- Comm  Need to list universities whose policies we referenced  Need to point to the role of consultants Schultz, Donovan, Howard, Skinner, 2010