Storage Why is storage an issue? Space requirements Persistence Accessibility Needs depend on purpose of storage Capture/encoding Access/delivery Preservation.

Slides:

Advertisements

Similar presentations

Data Publishing Service Indiana University Stacy Kowalczyk April 9, 2010.

Advertisements

Computer Basics Whats that thingamagige?. Parts of a computer.

Digital Collections: Storage and Access Jon Dunn Assistant Director for Technology IU Digital Library Program

Electronic Records Management Kevin Callaghan, Director Division of Records Management Tre Hargett, Secretary of State.

Secondary Storage Chapter 8 McGraw-Hill/Irwin

* Making Timely and Accurate Patient Care Decisions through Vista Imaging.

ARCHIVE IMAGING SEARCHABLE VIA THE WEBPAC Marthie de Kock The Hong Kong Institute of Education 9 December 2002.

Internet Resources Discovery (IRD) IBM DB2 Digital Library Thanks to Zvika Michnik and Avital Greenberg.

1 CS 502: Computing Methods for Digital Libraries Lecture 27 Preservation.

What is it? Hierarchical storage software developed in collaboration with five US department of Energy Labs since 1992 Allows storage management of 100s.

Storage Devices and Media

Designing Storage Architectures for Preservation Collections Library of Congress, September 17-18, 2007 Preservation and Access Repository Storage Architecture.

Computer Information Technology – Section 1-1. Parts of the Computer Objective: To identify the parts of a computer and their uses.

Standard Grade Computing STORAGE DEVICES CHAPTER 18 COMPUTER STUDIES Standard Grade.

Bits and Data Storage. Basic Hardware Units of a Computer.

Distinguish between primary and secondary storage.

HydraDAM2 NEH funding January 2015 – December 2017 Builds on existing WGBH HydraDAM Open source Hydra ‘head’ Preservation-oriented digital asset management.

Communications Technology 2104 Mercedes Lahey. Bit 1. bit=From a shortening of the words “binary digit” 2. the basic unit of information for computers.

Computer Storage & Representing Numbers CE 311 K - Introduction to Computer Methods Daene C. McKinney.

Comp 1001: IT & Architecture - Joe Carthy 1 Information Representation: Summary All Information is stored and transmitted in digital form in a computer.

Computer Systems I’m ONLY a machine! Standard Grade Revision.

Introduction to Computers

Svein Arne Brygfjeld National Library of Norway Nordic Web Archive.

Building a Fedora Architecture to Support Diverse Collections Jon Dunn Ryan Scherle Digital Library Program Indiana University.

Computer Systems Chapter 1 Pages Hardware-physical pieces Key hardware components in a computer system: The physical parts. – Central processing.

DLP Forum | March 31, 2003 Conversion Issues: Digitization Jenn Riley Digital Media Specialist IU Digital Library Program.

Reexamining Digital Library Infrastructure at IU Jon Dunn, Ryan Scherle, Eric Peters Indiana University Digital Library Program IU Digital Library Brown.

INFO1 – Practical problem solving in the digital world

Johannes Spitzbart Phonogrammarchiv, Austrian Academy of Sciences Österreichische Tage der Digitalen Geisteswissenschaften save the data - workshop on.

What is a Computer ? What is the application of computer in Our Daily Life ? What is the application of computer in Teaching Field?

Challenges of Digital Media Preservation Karen Cariani, Director Media Library and Archives Dave MacCarn, Chief Technologist.

OCLC Online Computer Library Center Digital Preservation with OCLC Digitization Standards: Issues & Updates Taylor Surface, OCLC.

 Computers store all information digitally as 0s and 1s; we refer to this collectively as “data.”  We’re used to dealing with physical objects in reality,

MULTIMEDIA DATABASES -Define data -Define databases.

Implementing a Data Publishing Service via DSpace Jon W. Dunn, Randall Floyd, Garett Montanez, Kurt Seiffert May 20, 2009.

Overview of Physical Storage Media

The Data Capacitor and Digital Libraries at IU Jon Dunn Associate Director for Technology IU Digital Library Program February 22, 2006.

Advanced IT to Support Digital Libraries Research and Academic Computing & Telecommunications Divisions UITS.

Chapter 7 Storage. What is storage? Holds data, instructions, and information for future use Storage medium is physical material used for storage Also.

VITAL at the National Library of Wales Glen Robson

Components of a Computer System

Sound Directions, 2007 IU, Harvard Best practices for audio preservation digitization and storage INDIANA UNIVERSITY MEDIA DIGITIZATION AND PRESERVATION.

Computer Organization. The Five Hardware Units General purpose computers use the "Von Neumann" architecture Also referred to as "stored program" architecture.

Floppy Disk Drive Lesson 5 CES Industries, Inc.. 1. Evolved from audio tape to floppy disk drives, with the first being an 8” disk to modern 3 1/2” 2.

2.1.4 Data Representation Units.

Storage devices 1. Storage Storage device : stores data and programs permanently its retained after the power is turned off. The most common type of storage.

Bits and Bytes. Overview  Bits - A binary decision  Byte - Most common unit of measurement for computer architectures (megabytes, mebibytes, gigabytes,

A Project of the University Libraries Ball State University Libraries A destination for research, learning, and friends.

File types and storage. Primary and Secondary storage  Primary – built into the computer, the computers memory capacity  RAM – random access memory.

Video Formats. 1.Students will understand what the difference is between analog and digital video. 2. Students will learn the advantages of digital video.

Discover ScholarSphere A repository service collaboration between the University Libraries and ITS.

WHAT IS CLOUD COMPUTING? Pierce County Library System.

© OCR 2016 Unit 2.6 Data Representation Lesson 1 ‒ Numbers.

STORAGE DEVICES Towards the end of this unit you will be able to identify the type of storage devices and their storage capacity.

Computer basics.

Part B Computer Storage

Memory Parts of a computer

STORAGE DEVICES Towards the end of this unit you will be able to identify the type of storage devices and their storage capacity.

VI-SEEM Data Repository

Flexible Extensible Digital Object Repository Architecture

Flexible Extensible Digital Object Repository Architecture

VI-SEEM Data Repository

Research Data Archive - technology

STORAGE DEVICES Towards the end of this unit you will be able to identify the type of storage devices and their storage capacity.

Jon W. Dunn Assistant Dean for Library Technologies

How do computers work? Storage.

IBM Archiving Solutions for SAP

The Bentley Digital Media Library

Presentation transcript:

Storage Why is storage an issue? Space requirements Persistence Accessibility Needs depend on purpose of storage Capture/encoding Access/delivery Preservation

Storage: Working Space Space for storage of digital files during capture/encoding/quality control process Possibilities PC hard drive File server, e.g. marengo (LIT) DLP file server Issues Capacity, backup, speed, accessibility

Storage: Access/Delivery Storage for web delivery of images, audio, text, etc. Possibilities UITS web server, under library account UITS streaming media server (audio/video) DLP web server Issues: capacity, backup, performance, software integration, maintenance/migration

Storage: Preservation Much harder problem Longer term Issues of longevity of media, hardware, file format Where are the files? Larger files Hard disk storage, traditional backup methods not cost-effective Infrequency of access Problems do not become immediately evident

Long-Term Storage Options Removable media e.g. CD-R, DVD-R Pros: cheap, easy, produces tangible item Cons: Low capacity, physical space requirements, unknown longevity, migration Nearline storage UITS Massive Data Storage Service

UITS MDSS Massive Data Storage Service HPSS (High Performance Storage System) software Developed as collaboration of IBM and US national labs Four tape robots (two at IUB, two at IUPUI) Data can be mirrored 540 TB total storage ~75 TB used as of April 2001

MDSS – A Sense of Scale 2 Kilobytes : A typewritten page 5 Megabytes : Complete works of Shakespeare OR 30 seconds of TV quality video 1 Gigabyte (1000MB) : 1 pickup truck filled with paper OR a symphony in hi-fi sound 1 Terabyte (1000GB) : All the X-ray films in a large hospital OR paper from 50,000 trees 10 Terabytes : The printed collection of the US Library of Congress 50 Terabytes : The contents of a large mass store system 8 Petabytes (8000TB) : All information available on the web 200 Petabytes : All the printed material (in the world!)

MDSS Storage Infrastructure

MDSS Access FTP/PFTP: (Parallel) File Transfer Protocol DFS: Distributed File System (being phased out) HSI Not practical for delivery Hierarchical storage (metadata on disk, data on tape -> second to start transfer.) File size – chunks of 50 MB or greater work best Small files aggregated into larger.tar or.zip files

DL Objects Digital library “objects” have many parts Metadata Preservation files Delivery files How do we keep them connected? Now: Good practice in file naming, directory organization, project documentation -not scalable! Future: Digital object repository

Data Persistence Key is migration Keeping the bits alive - MDSS responsibility Physical media Logical media format Keeping the bits understandable - MDSS user responsibility File format Metadata Small “pockets” of digital content pose a problem for migration

DL Object Repository Preservation version in MDSS Delivery version on web server Metadata records Repository System Users and applications

DL Repository Models OAIS: Open Archival Information System Reference model Fedora: Flexible and Extensible Digital Object and Repository Architecture Developed at Cornell and UVa IU DLP in deployment group

DLP Storage Services Consulting Server space for production and access Persistent naming service (PURL server) Facilitation of access to UITS services Streaming media MDSS Developing repository service Contact: