1 Introduction to Archiving Movies in a Digital World Dave Cavena, Sun Microsystems January, 2007

2 Agenda Overview Archiving Archived content integrity Proposed model Costs Alternatives? Summary Conclusion

3 Overview Has the time come to begin archiving movies digitally? Only archiving remains reliant on film Digital image archive technology is mature A viable, scalable, cost-effective COTS model What are the alternatives?

4 The stories of an Age Fiduciary responsibility A Digital Content Archive can store these assets without degradation forever Archiving

5 Chairman Vice Chairman Archiving Any movie archived in 1907 is playable in 2007 Will a celluloid movie archived in 2007 be playable in 2107? Is it time to start digital archiving of this irreplaceable content?

6 Archiving Will the Archive be the only time the story exists on film? What are celluloid archive and repurposing costs? A Digital Content Archive provides image and cost advantages over celluloid Can be accomplished with COTS Technology

7 Archived Content Integrity Irreplaceable content Multiple copies Multiple libraries Automated audit, copy Algorithmic assurance of bit integrity Error Correction Codes (ECC) Bit Error Detection Bit Error Correction

8 Archived Content Integrity ECC Standard on tape drives COTS technology Bit Error Rates* Bit Error Rates (BER) differ by manufacturer ECC undetected BER = Four copies = ECC uncorrected BER = Four copies = TB Digital Intermediate = bits One uncorrectable bit error in movies ( * ) * Sun T10000 drive

9 Archived Content Integrity Generational data integrity 20 generations of compute/disk front-end 5 generations of libraries Unknown generations of application file formats At least 12 rewrites of the content onto new media What is the generational impact on the algorithmic BER?

10 Archived Content Integrity For this application it doesn’t matter how many times the data is accessed; how many generations of rewrite Probability that the ECC will fail to correct damage during any given access is The probability it will fail one or more times during N accesses is 1 minus the probability that it will succeed N times in a row: 1-( )N For N less than 10 19, this is well approximated by N*10 -19

11 Archived Content Integrity Example Assume a movie accessed one million times The chance of an uncorrectable bit error per read is The chance of an uncorrectable bit error on any one of 10 6 reads is 10 6 * = For a single copy It reasonably can be assumed for the purposes of this application that the ability to detect and correct errors in transcription is perfect.

12 Archived Content Integrity Other Strategies Secure Hashing Algorithm, SHA-256* Checksum failure probability of , or approximately Four-copy BER = One undetected bit loss in movies Birthday collisions don’t apply; not defending against traffic analysis, just using it as a good checksum Voting bit-by-bit Can make a 10TB DCDM into 40 1TB files, 31 of which would have to be damaged to preclude rebuilding the original * Developed by the NSA, publicly available, peer-reviewed, easy to implement

13 Archive Model Enterprise class tape library Front-end server and disk Ingest and prepare Archive Object for writing to tape library Hierarchical Storage Manager, HSM Two complete and identical systems, geographically separate Two copies of each movie on each library

14 Archive Model Computers and disk front-ends reach EOSL 5-yr replacement Tape drives reach EOSL 10-yr replacement Libraries reach EOSL 20-yr replacement Tape media has a finite lifetime* Replace tapes every 10 years Audit every tape every six months Re-write from pristine copies as necessary *National Media Lab, IBM, Sun, others, publish 30 years as viable tape media lifetime

15 Archive Model Application software and file formats Proposed archive model HSM uses an open tarball format, readable even without the application When a tape is audited, rewritten or copied, the new copy can be created in the new file format This is feasible because the underlying data format remains digitally fixed, only the file format and / or storage medium change

16 Archive Model Institutional memory must be created Two or more sites are required, geographically separate No network connectivity Archive content in the clear Same as current model Lost key or algorithm will render archive useless Can be encrypted for transport (tape drive HW encryption becoming the norm) When copying tapes, send old ones to another location

17 Oil & Gas has been archiving digital images for decades Medical is doing this with far higher transaction rates Library of Congress doing it now "Storing National Treasures" "Sun Rises at the Library of Congress" Archive Model

18 Costs Can digital compete with celluloid? Film archiving cost $100K /100 years / feature 2,000 movies = $200M 10TB archive object, 20 objects/year, 100 years $45,000/movie (list) $16,000/movie (Archive pricing) 2,000 movies = $32M 100TB archive object $67,000/movie (Archive pricing) 2,000 movies = $79M

19 Costs 10TB Archive Object – List price $2,601,160 $408,631 $114,218 $73,094 $57,330 $45,493 $0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 Movies in Archive Dollars

20 Costs 10TB Archive Object – List price SAM License + mtce75% Media 3% Library and Drives + mtce 18% Compute/Disk + mtce 4% Description (both libraries, two copies/movie/library)Cost Compute/Disk + mtce$3,200,000 Library and Drives + mtce$16,112,000 Media$2,699,000 SAM License + mtce$68,974,400 Total $90,985,400

21 Costs 10TB Archive Object – Archive price

22 Costs 10TB Archive Object – Archive price $2,005,850 $236,852 $54,072 $29,706 $21,259 $16,265 $0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 Movies in Archive Dollars

23 Costs 100TB Archive Object – List price $6,488,535 $5,006,303 $3,791,715 $3,180,640 $2,641,742 $2,121,252 $0 $1,000,000 $2,000,000 $3,000,000 $4,000,000 $5,000,000 $6,000,000 $7,000,000 Movies in Archive Dollars

24 Costs 100TB Archive Object – List price

25 Costs 100TB Archive Object – Archive price

26 Costs 100TB Archive Object – Archive price $1,462,466 $505,009 $170,432 $112,159 $85,527 $67,171 $0 $200,000 $400,000 $600,000 $800,000 $1,000,000 $1,200,000 $1,400,000 $1,600,000 Movies in Archive Dollars

27 Alternatives An unaddressed question… Does celluloid have a future – at all? Replaced by commercial photographers globally Precipitous drop in market share and manufacturer jobs Environmentally unfriendly to manufacture and process Celluloid may not be an option Film may not even exist in 100 years Film infrastructure – labs, chemicals, workers, etc. - may not exist

28 Summary The technology required to store and maintain irreplaceable digital image content for archive durations is mature, proven and in use today A Digital Content Archive will extend the quick responsiveness of a studio’s Library to the Archive The return on these increasingly expensive assets easily can be extended – forever … all using COTS technology

29 Conclusion The pivotal and immutable point is that this can be done beginning today. The experience Sun brings to the project already has been recognized, and is being broadened by, the Library of Congress and other locations around the world undertaking the digitization of their media assets using solutions from Sun Microsystems. The time is now to begin serious efforts to test and implement studio Digital Content Archives

30 Introduction to Thank you Dave Cavena