Presentation on theme: "Mairéad Martin, Penn State University Commons Solutions Group Storage Workshop May 2010."— Presentation transcript:
Mairéad Martin, Penn State University Commons Solutions Group Storage Workshop May 2010
Designing and implementing storage architectures and systems to support data curation and preservation needs ◦ What does this entail? ◦ Who’s thinking about this? ◦ Who’s doing anything about this?
Digital Preservation ◦ Managed activities to ensure long term retention, retrieval of, and access to data Digital Curation ◦ Maintaining, preserving, and enhancing data throughout its lifecycle Archival storage ◦ Depends on who you talk to Information Lifecycle Management ◦ Storage industry term for the above Object-based storage ◦ Data with metadata “container”
eScience/eResearch data management needs NSF requirement for data management plans Compliance ◦ e-Discovery, FERPA, HIPAA, Sarbanes-Oxley ◦ Institutional record retention regulations and policies Storage services for libraries, archives, cultural heritage entities Great efficiencies
Storage is cheap Storage is smart Stuff on the Internet is persistent Digital safer than analog Storage provider = curators and preservation experts Repositories take care of preservation Metadata will take care of it Libraries will take care of it The Cloud will take care of it
New roles, new responsibilities, new collaborations, practices, workflows Intellectual capital requirements – digital preservation/curation policy determination and implementation Bar for trust is rising Cloud antithetical to preservation? Increased storage management requirements Scaling issues with preservation requirements
More likely to meet these today at the system level – DR & BC practices and tiered storage architectures Immutable storage Data integrity checking ◦ Mitigation of bit rot ◦ Auditing function Mitigation of obsolescence ◦ File format migration Deposition as important as retention Need for storage management metadata ◦ Technical – file size, name, location, ACL, date, time, versioning, Biggest need: system-independence
iRODS (integrated Rule-based Data System) Storage Resource Broker (SRB) Content Addressable Storage (CAS) ◦ Fixed content storage, retrieval based on content rather than location eXtensible Access Method (XAM) ◦ Emerging SNIA standard for an API for content- addressable storage objects
NSF DataNet Program ◦ Data Conservancy project – JHU lead with 23 institutions to create curation, discovery, and preservation network Chronopolis ◦ SDSC, UCSD, UMIACS, NCAR: Federated data grid using SRB/iRODS LOCKSS (Lots of Copies Keep Things Safe) ◦ Replication of licensed journals and other content MetaArchive – ◦ a private LOCKSS archive Internet Archive
National Digital Information Infrastructure & Preservation Program (NDIIP) ◦ Library of Congress program to “to develop a national strategy to collect, preserve and make available significant digital content via a preservation network of over 130 partners."
California Digital Library ◦ Curation Micro-services DuraSpace ◦ DuraCloud project to implement a preservation- oriented cloud storage service HaithiTrust ◦ Repository and storage infrastructure initiated for CIC Google book project Sun Preservation and Archiving SIG (PASIG) Storage Networking Industry Association
Content Stewardship Program – strategic collaboration between University Libraries and Information Technology Services (ITS) Goal: a suite of services to support the lifecycle of the digital object – creation, discovery, access, storage, preservation and archiving Hired Digital Library Architect and Digital Collections Curator Governance in place
Anchor projects/activities: ◦ Storage and Preservation strategy development Prototyped the XAM standard for archival storage ◦ Institutional record repository ◦ Research data prototype ◦ Best practices for data management ◦ ETD platform replacement Sponsoring curation technology workshop in August LOCKSS member, recently joined MetaArchive Exploration of California Digital Library’s curation micro-services Application of service management principles and processes to the above
What are CSG member institutions doing in this space?