1. TiBS Fermilab – HEPiX-HEPNT Ray Pasetes October 22, 2003

2. Problem AFS full backups > 40 hours UNIX full backups > 20 hours Operator assisted tape mounts Mandate for automated operations

3. Investigation March – July 2003 Software evaluated included –Veritas Netbackup –SyncSort –IBM/Tivoli TSM –Amanda –Teradactyl/TiBS

4. Criteria Support for IBM AFS and OpenAFS Reduce backup windows < 24 hours Complete automation Scalable Reliable Affordable

5. Evaluation AFS/ OpenAFS < 24 Window Automated ScalableReliableAffordable Veritas NoYes No SyncSort NoYes No IBM Yes No Amanda Yes No* TiBS Yes

6. Notes: Veritas No support for OpenAFS No ongoing development for AFS No bug fixes for current AFS offering Pricing –Per seat, by OS, #CPUs, and CPU type –Number of tape drives Different pricing for SAN attached drives –Lots of add-on feature charges

7. Notes: SyncSort No support for AFS Pricing –Per seat, UNIX or Windows –Number of tape drive slots in library Different pricing for SAN attached drives –Pricing came in packs May not fit current equipment. I.E. 170 tape slot pack for 120 slot library 25 pack UNIX licenses for 18 UNIX systems

8. Notes: IBM Actively supporting AFS and OpenAFS Can do single file restores for AFS Pricing –Vague pricing unless we were willing to commit to significant purchase –Per seat, OS, CPU type

9. Notes: Amanda AFS support enhanced by internal work Pricing –The support effort to maintain the product was not available

10. Notes: TiBS Support for AFS and OpenAFS Pricing –Client Per OS, irregardless of clients Enterprise (all supported OSes) –Server Per process (number of parallel backups)

11. Decision July 2000, decision to deploy TiBS Production -- August 2000 –400GB of data, all AFS –Only part of CSI backups migrated Currently, ~1.6TB – 140+ total systems AFS, HFS+, NTFS, UNIX file systems. –IRIX, Linux, OSX, Solaris, Tru-64, W2K, XP –Backups for CSI and SCS migrated Backup Window ~< 5 hours

12. TiBS: How does it work? TiBS == True incremental Backup System First backup is a full backup over the network All subsequent backups are incrementals over the network Subsequent weekly and full tapes generated from previous Fulls, Weeklies and incrementals at the server, without client participation All data staged to spooling disk first

13. More on TiBS Incremental data –Mirrored in disk cache –Merged with previous incrementals daily Does allow for direct-to-tape backup –Can not use merge process One-pass restore

14. Additional Benefits AFS –Reports on volumes that need salvaging –Consistency check between vldb and fileservers –Look up individual AFS files before restoration General –Hostaudit reports any new partitions/disks on a backup client –Merge process auto-checks integrity of tapes –Lost tapes can be reconstructed from previous tapes –User initiated backups -- laptops –Support has been excellent

15. Cons May require large caching disk –Typically 10% of total data backed up –Cache needs to be at least as large as largest filesystem Requires a larger library –Or more aggressive tape handling –Reliance on previous tapes to create data

16. Keep in Mind Non-traditional scalability factors – Size of individual filesystems vs. overall storage –Disk/cache and tape speed vs. network speed –CMU-CS backs up 7TB using a single sparc, 512MB memory, 280GB cache, 4DLT8Ks. Higher reliance on previous backups Failed backups have higher significance

17. More on Teradactyl Roots are from CMU –Birthplace for AFS –CMU ECE and H&SS departments are beta sites Very open to customizing solutions –K5/KCA authentication

18. Summary Originally purchased for CSI AFS backups and UNIX server backups (20 systems total) Proved to be very scalable on current hardware and has expanded to two groups, 140+ systems We are now backing up desktops, laptops, Windows and Mac OSX as well as AFS and traditional UNIX systems Does not require enterprise version of Linux OS

19. Additional Information FNAL –http://www-oss.fnal.gov/csi/afs/stats/tibs/html/bindex.htmlhttp://www-oss.fnal.gov/csi/afs/stats/tibs/html/bindex.html –Ray Pasetes rayp@fnal.gov –Kevin Hill kevin@fnal.gov Vendor –http://www.teradactyl.comhttp://www.teradactyl.com