© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 Backing Up for Eternity Dr. Martin Atkins Parvat Infotech Private Limited

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 Why don't people do backups? ● They don't understand how to do it – in a reasonable time, given their setup, etc. ● They don't get any added value from backups – until the system breaks! ● They don't like 'wasting' media – I only wrote this CD last week.... ● They don't think their data is so important – right up until they lose it! ● They have better things to do

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 But I do do backups! ● Most backup systems – are designed primarily to backup to tapes ● tapes are re-cycled (losing history) ● tapes and tape drives are expensive, awkward, unreliable, etc. – can backup to write-once media, but: ● each backup supersedes much of the old backup ● old media becomes almost useless (beermats!) ● no help for finding historical copies of data

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 Eternal - Management Overview ● Set-up and forget – as far as possible ● Not just backup - also history – What did that file look like yesterday?... Last week?... Last year?... – Eternal history filesystem mounted on /backup – e.g. cd /backup/2003/aug/21/home/martin/... ● Old archive media is never superseded – it becomes part of your history record

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 How is this possible? ● Identical data blocks are never stored twice – uses the 'Venti' algorithm ● developed for Plan 9 (from Bell Labs) – tremendous 'compression', e.g. ● every machine in a network has the same copy of vmunix, etc ● most of my files are the same today, as they were yesterday – makes it feasible to store the total system and/or network history

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 Techie Overview ● Client / Server architecture – both can be on the same machine ● No Full / Incremental backup decision – all backups are created equal – every backup contains the entire filesystem state ● Primary operation is 'disk-to-disk' – backups can be fully automatic – restore operations do not need access to removable media

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 Friendly towards removable-media ● data is batched in CD (or DVD) sized units ● media can be written when it is convenient for you! ● data, once written, is never changed – 2 copies can be made for extra security ● 1 for your shelf ● 1 to go off-site (in a bank vault?) ● media is only read if backup server must be restored

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 How does it work? ● Each block of data is indexed by its 'fingerprint' – the 160-bit SHA1 hash of the contents – identical data blocks will hash to the same value – different data blocks can also hash to the same value ● but only with a probability of 1 in (i.e. 1 in ) ● if we store 10 6 terabytes, the chance of an identical hash somewhere in the data is about 1 in ● SHA1 is 'cryptographically secure' – meaning that 'you can't calculate it backwards in a reasonable time'

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 The Server ● Server receives request to store a block 1) calculates the block's fingerprint 2) if that finger print not in index: store block in the backup history 3) return the block's fingerprint ● Server receives request to retrieve a block 1) fingerprint given in request 2) if fingerprint is not in index return error 3) return data from backup history

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 The Client ● E.g., backup /d1/d2/f1 /d1/d2/f2 /d1/f3 ● tar-like utility files  20 bytes f1 f2 f1: f2: d2: f3: D1: f3 f1 f2 f1: f2: d2: f3: D1: f3 f3' d2: f3: D1': Monday Tuesday fingerprint: data block: Key:

© 2003, Parvat Infotech Private Limited Linux Bangalore / 2003 Status and Information ● Eternal is currently being developed by Parvat Infotech in Bangalore ● first release intended for early 2004 ● try-before-you-buy licenses – individual and commercial prices ● For more information, please see