1. Computing Infrastructure
Hardware & Software
GrADS
Project Athena Technical Issues
Computing infustructure
computing facilities
grads
project athena technical issues

2. IGES Computing Requirements
Large model runs done offsite
Requires high bandwidth connectivity to remote sites
In house focus on data analysis
Must provide shared access to data
Requires large disk space
Compute servers for analysis

3. Network Public Internet Internet 2 Firewall for all networking
T-3 ~ 20Mb/s
Internet 2
1Gb/s
Firewall for all networking
Internal
Gigabit Ethernet
Infiniband

4. Servers Dell hardware (linux) Public servers
Front end servers (cola##)
8 cpu
32GB ram
Public network
Compute servers (cpu#)
24-32GB ram
Private network

5. Data storage Dell Intel servers (linux) with direct attached disk
Disks in RAID 6
Dual drive redundancy
Gluster file-system
High speed clustered file-system
Open-source
Data is stored on standard Linux file-systems (XFS)
Shared over Infiniband network

6. Data access Gluster 6 servers No metadata server 100% scaling
24 disk bricks
No metadata server
100% scaling
~2 GB/s aggregate throughput
~400 MB/s single stream

7. IGES DATA accessibility
Single name space for storage, 191TB
Mounted to every server
All users have access to all data
Saves on duplicate data sets
Users on any server
/homes
/data
Single namespace

8. Reliability / Backups No single point of failure Bacula for backups
Dual drive redundancy
4 hour or NBD support contracts
Spare parts onsite
Bacula for backups
All data backed up weekly
500 Slot Quantum i2000 tape loader
3 LTO-4 tape drives
2 copies of everything
Up to 3000 slots, 96 drives

9. Future plans - hardware
Data archiving
2 copies of data to tape
Increased disk storage
CPU servers with more cpu cores and large amounts of memory analysis
24 cores / 128 GB memory

10. Software stack Open source software whenever possible
Servers run identical software
Example open source packages R
Flint
On each server
Operating system
Compilers
GrADS
Home directory
Data directory R
Flint

11. Questions…