Ray Pasetes, Andy Romero CSS Storage Services CD/CSS Central Services Activities Coordination Meeting 9/19/2006

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services Storage Services - Priorities 1. Reliability A.Data integrity B.Service availability C.Manageability 2. Cost 3. Performance

September 2006CSS Storage Services Storage Services Tier 1 RAID 5 Fibre-Channel Tier 2 RAID 5/6 SATA Fabric-AFabric-B AFS Global Access <25MB/s NFS/CIFS Campus Access ~ MB/s SCSI Local to FCC >200MB/s per lun Backups IP FCP NDMP SAN

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services Tiered Storage Pool Redundant components Redundant components Tier 1 - RAID 5 Fibre Channel disk Tier 1 - RAID 5 Fibre Channel disk –High I/O applications –Single disk fault tolerant Tier 2 – RAID6 SATA disk (some RAID5) Tier 2 – RAID6 SATA disk (some RAID5) –Low to medium I/O applications –Double disk fault tolerant

September 2006CSS Storage Services Direct SAN Attach - FCP Block-based access (SCSI) Block-based access (SCSI) Must be in FCC (500m limit) Must be in FCC (500m limit) 2Gbps Fibre-channel network 2Gbps Fibre-channel network Multi-attach for 24x7 service Multi-attach for 24x7 service Support for Linux, Solaris, IRIX, Windows Support for Linux, Solaris, IRIX, Windows

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services NAS Services - AFS Global filesystem Global filesystem Throughput < 25MB/s Throughput < 25MB/s Secure Secure –Communication can be encrypted –Kerberos authentication –Directory ACLs Limited to 8GB volumes Limited to 8GB volumes

September 2006CSS Storage Services NAS Services – NFS/CIFS Campus-wide access only Campus-wide access only –CIFS off-site via VPN possible Throughput Throughput –~200MB/s write –~300MB/s read HA cluster HA cluster Many virtualization services Many virtualization services

September 2006CSS Storage Services NAS Service Overview 2 Titan-21 NAS Gateways (Physical) 2 Titan-21 NAS Gateways (Physical) 2 Enterprise Virtual Servers (EVS) 2 Enterprise Virtual Servers (EVS) –Each EVS has its own IP address –Each EVS provides NFS, CIFS and FTP services –An EVS can run on either Titan (HA Failover Capability) All storage provided by SAN All storage provided by SAN 4 Gb/s connection to 256 Gb/s LAN core 4 Gb/s connection to 256 Gb/s LAN core –high performance for all customers

September 2006CSS Storage Services NAS Overview Storage Pools Storage Pools –Composed of LUNS (units of physical storage provided by SAN) File Systems File Systems –Multiple file systems reside in a pool –Different block sizes (large or small files) Virtual Volumes (ViVols) Virtual Volumes (ViVols) –Multiple ViVols reside in a pool –Key to thin provisioning

September 2006CSS Storage Services NAS Virtualization Services Zero downtime capacity upgrades Zero downtime capacity upgrades Thin Provisioning Thin Provisioning Snapshots Snapshots Policy Based Data Migration Policy Based Data Migration

September 2006CSS Storage Services Capacity Upgrades DAS Capacity Upgrade DAS Capacity Upgrade –Move server to a new rack to fit new chassis –Install new RAID channels to support new RAID chassis –Install new RAID chassis –Create new file systems –Migrate / Rebalance data –Downtime (Lots !!)

September 2006CSS Storage Services Capacity Upgrades NAS Capacity Upgrade NAS Capacity Upgrade –Increase size of Vivol (best case) –Add LUNs to storage pool (worst case) –No need to move server –No need to install new RAID channels –No need to create new filesystems –No need to migrate and rebalance data –Downtime (Zero !!)

September 2006CSS Storage Services Storage Provisioning DAS Style Storage Provisioning DAS Style Storage Provisioning –Each Project/Dept has its own server –Each server has its own disks –Each server has its own free space pool –Storage provisioning predictions are often wrong Some projects, plenty of available excess (sum of total free space across all servers TB+) Some projects, plenty of available excess (sum of total free space across all servers TB+) Others, heavily used…starved for space Others, heavily used…starved for space New project = new disk purchase New project = new disk purchase –Sorry …. Sharing not possible

September 2006CSS Storage Services Storage Provisioning NAS/SAN Style Storage Provisioning (Thin) NAS/SAN Style Storage Provisioning (Thin) –All Projects/Services share the same SAN connected storage –ViVol parameters can be tuned to over provision physical storage –New Project, no problem Get started now Get started now Defer storage purchase until later Defer storage purchase until later –Easier to defer and lump storage purchases Better Prices Better Prices

September 2006CSS Storage Services Snap Shots …\Live-Files …\.snapshot

September 2006CSS Storage Services SNAP Shots Why snapshots are cool Why snapshots are cool –User’s can recover their own data immediately –No need to submit helpdesk ticket

September 2006CSS Storage Services Policy Based Data Migration (PDM) Ability to automatically migrate data from expensive tier 1 to tier 2 disk Ability to automatically migrate data from expensive tier 1 to tier 2 disk Does not interrupt users Does not interrupt users Various Migration Criteria Various Migration Criteria –Age –Size –Type (avi, jpg, tiff …..) Various Analysis Schedules Various Analysis Schedules

September 2006CSS Storage Services NAS Status Windows Windows –Backups, Printer Migrations –Soon CSS Home directories CSS Home directories PSEEKITS soon PSEEKITS soon VMSSTREAMER (data only) VMSSTREAMER (data only) CDSERVER (printers first) CDSERVER (printers first) PPDSERVER (printers first) PPDSERVER (printers first) VMSSERVER1 (printers, www services first) VMSSERVER1 (printers, www services first) Others … Others …

September 2006CSS Storage Services NAS Status NFS NFS –CMS, OSG, Scientific Linux –Many customers, Some unexpected growth (…new NAS on order) Can I use the NAS for my project Can I use the NAS for my project –YES … It’s a site service –Costs Small projects …. None Small projects …. None Large projects …. share portion of disk, tape costs (no management costs) Large projects …. share portion of disk, tape costs (no management costs)

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services Virtualization Storage Pool Connection Access Strategic Direction RAID Storage Automated Tiering Thin Provisioning TCP/IP FCP SnapshotsNAS File Protocol NFS/CIFS/FTP ~200MB/s Campus wide SAN Block Protocol SCSI-3 >200MB/s Inside FCC Thin Provisioning Automated Tiering Snapshots FCP or TCP/IP Dynamic ResizeDynamic Tuning

September 2006CSS Storage Services Additional Information Questions?

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services Backups TiBS – True Incremental Backup System Only 1 full network backup Only 1 full network backup All other backups are incremental All other backups are incremental –Lightens client workload Use previous backups to create new fulls Use previous backups to create new fulls No “per-client” connect costs No “per-client” connect costs

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services TiBS – Supported Systems Servers Servers –IRIX, Linux, Mac OSX, Solaris, Windows Client code must be installed Client code must be installed –NAS appliances via NFS

September 2006CSS Storage Services TiBS – Unsupported Sparse Files (databases) Sparse Files (databases) –Needs to be dumped to flat file Compressed files as filesystem Compressed files as filesystem Swap files Swap files Desktops Desktops Raw devices/block-level backups Raw devices/block-level backups

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services TiBS – How it works Tue Full Mon M-T Merge MonWed M-T-W Merge Previous Full New Full Merge Disk Cache

September 2006CSS Storage Services TiBS – Tape Rotation Full - 84 days (mirrored) Full - 84 days (mirrored) –Retained 1 year Monthly – 28 days (mirrored) Monthly – 28 days (mirrored) –Retained 1 year Weekly – 7 days Weekly – 7 days –Retained 14 weeks Incremental – Daily Incremental – Daily –Retained 56 days**

September 2006CSS Storage Services TiBS - Recoverability 1 Full tape lost 1 Full tape lost –Use other mirror to recover Both Full tapes lost Both Full tapes lost –Use current monthly and previous full to recreate full Both Full tapes and both month tapes lost Both Full tapes and both month tapes lost –Use current weekly and previous month to recreate new current month –Use new current month with previous full to recreate new current full

September 2006CSS Storage Services TiBS – Actual size of data Multiple copies of data on tape Multiple copies of data on tape –Mirroring –Retention policies –For one year, size ratio is 14:1 Tape vs. disk for static data. More for dynamic data. Tape vs. disk for static data. More for dynamic data.

September 2006CSS Storage Services Agenda SAN/NAS services 1.Priorities 2.SAN 3.NAS 4.Strategic Direction Backups - TiBS 5.Supported 6.How it works 7.Stats

September 2006CSS Storage Services Stats and Facts 415 servers being backed up 415 servers being backed up –BSS, CMS, CSS, DCG, DIR, DSG, ESE, ESH, EPP, FESS, MIPP, QCD, REX 6770 AFS volumes backed up 6770 AFS volumes backed up ~15 TB of data == 1 full backup ~15 TB of data == 1 full backup Clients can recover from bare metal* Clients can recover from bare metal* Limited availability for new customers Limited availability for new customers –Tape resources

September 2006CSS Storage Services More Information Questions?