1. IDE disk servers at CERN Helge Meinhard / CERN-IT CERN OpenLab workshop 17 March 2003

2. Introduction HEP computing in the past: mostly reading from, processing, and writing to (tape) files sequentially Mainframe era (until ~ 1995): single machine, CPUs, tape drives, little disk space In response to scaling problem, development of SHIFT architecture (early 1990s) Scalable farm out of ‘commodity’ components RISC CPUs (PowerPC, MIPS, Alpha, PA-RISC, Sparc) SCSI disks

3. SHIFT architecture Network - FDDI - Hippi - Myrinet - Ethernet Tape server Disk server Batch server Interactive server Batch and disk SMP Network - Ethernet

4. PC batch nodes 1995: First studies at CERN of PCs as batch nodes (Windows NT) 1997 onwards: Rapidly growing interest in Linux (on IA32 only) 1998/99: First production farms for interactive and batch services running Linux on PC hardware at CERN

5. PC disk servers 1997/98: Prototypes with SCSI disks 1998/99: Prototypes with EIDE disks Different IDE adapters Not RAIDed 1999/2000: First Jumbo servers (20 x 75 GB) put into production 2001: First rack-mounted systems 2002: 97 new servers (54 TB usable) 2003: 1.3 TB usable in one server at 13 kCHF Total usable capacity today: ~ 200 TB

6. 1997: ~700 GB SCSI/Sparc

7. 2000/2001: 750 GB PC/EIDE (1)

8. 2000/2001: 750 GB PC/EIDE (2)

9. 2002: 670 GB PC/EIDE 2 systems

10. Disks only Complete systems SCSI/ RISC EIDE/ PC

12. Gross Usable

13. Today’s servers: Specifications 19” rackmount, IA32 processor(s), 1 GB, 2x80 GB system disks, GigE (1000BaseT), redundant power supplies >500 GB usable space on data disks Hardware RAID offering redundancy Hot-swap disk trays Performance requirements network – disk: 50 MB/s reading from server @ 500 GB 40 MB/s writing to server @ 500 GB 5 years on-site warranty

14. Lessons learnt Capacity is not everything, for good performance need CPU, memory, RAID cards Good OS and application software Network connectivity Large number of spindles Firmware of RAID controllers and disks critical Redundancy (RAID) is a must, required performance possible only with mirroring (RAID 1) so far

15. Outlook Good price/performance has risen interest in other application domains at CERN AFS and MS DFS servers Web servers, mail servers Software servers (Linux installation) Data base servers (Oracle, Objectivity/DB) Access pattern of physics analysis likely to change Investigating different file systems (XFS), RAID 5 (in software), … Architecture constantly being reviewed Alternatives investigated: data disks scattered over large number of batch nodes; SAN

16. Conclusion Architecture of early 1990s still valid May even carry us into LHC era… Important improvements made Price/performance Reliability (RAID) Will review architecture soon (2003) New application areas New access patterns for physics analysis