1. Presented by NCCS Hardware Jim Rogers Director of Operations National Center for Computational Sciences

2. 2 Rogers_NCCS_Hardware_SC07 4512 quad- core processors @ 2.3 GHz 18,048 GB Memory TRACK II 2048 quad- core processors @ 850 MHz 4096 GB Memory BLUE/GENE PIBM LINUX NSTG (56) 3 GHz 76 GB Memory 4.5 TB VISUALIZATION CLUSTER (128) 2.2 GHz 128 GB Memory 9 TB CRAY XIE PHOENIX (1,024) 0.5 GHz 2 TB Memory 44 TB NCCS resources Network routers UltraScience 10 GigE 1 GigE Control network 7 Systems October 2007 summary Scientific visualization lab Supercomputers 51,110 CPUs 68 TB Memory 336 TFlops Total shared disk 250.5 TB 5 PB data storage 35 megapixels Powerwall Test systems 96-processor Cray XT3 Evaluation platforms 144-processor Cray XD1 with FPGAs SRC Mapstation Clearspeed BlueGene (at ANL) 5 PB Backup storage (23,662) 2.6GHz 45 TB Memory 900 TB CRAY XT4 JAGUAR IBM HPSS 5 TB Many storage devices supported

3. 3 Rogers_NCCS_Hardware_SC07 Hardware roadmap As it looks to the future, the National Center for Computational Sciences expects to lead the accelerating field of high-performance computing. Upgrades will boost Jaguar’s performance to 250 teraflops by the end of 2007, followed by installation of two separate petascale systems in 2009.

4. 4 Rogers_NCCS_Hardware_SC07 Jaguar system specifications 54 TF100 TF250 TF1.000 TF Compute processors 5,212 Dual-core 2.6 GHz Opteron 11,508 Dual-core 2.6 GHz Opterons 7,816 Quad-core Opterons 24,000 Multi-core Opterons SIO processors 82 Single-core 2.4 GHz Opteron 116 Dual-core 2.6 GHz Opteron 124 Dual-core Opteron 544 Quad-core Opteron Memory per socket/total 4 GB / 20 TB total system 4 GB / 45 TB total system 8 GB / 63 TB total system 32 GB / 768 TB total system Interconnect bandwidth per socket Seastar 1 1.8 GB/s Seastar 2 4.0 GB/s Seastar 2 4.0 GB/s Gemini Disk space 120 TB900 TB 5-15 PB Disk bandwidth 14 GB/s5 GB/sTotal 55 GB/s240 GB/s

5. 5 Rogers_NCCS_Hardware_SC07 Phoenix – Cray X1E  Ultra-high bandwidth  Globally addressable memory  Addresses large-scale problems that cannot be done on any other computer CRAY X1E: 1,024 vector processors, 18.5 TF

6. 6 Rogers_NCCS_Hardware_SC07 Today: Jaguar – Cray XT4  120 TF Cray XT4  2.6 GHz dual-core AMD Opteron processors  11,508 compute nodes  900 TB disk  124 cabinets  Currently partitioned as 96 cabinets of catamount and 32 cabinets of compute node linux  November 2007: All cabinets compute node linux

7. 7 Rogers_NCCS_Hardware_SC07 Cray XT4 scalable architecture Jaguar – Cray XT4 architecture  Designed to scale to 10,000s of processors  Measured MPI bandwidth of 1.8 GB/s  3-D torus topology

8. 8 Rogers_NCCS_Hardware_SC07 Phoenix – Cray X1E Phoenix Utilization by Discipline 100 OctDecFebAprJunAug Utilization (%) 90 80 70 60 50 40 30 20 10 0 Astro Atomic Biology Chemistry Climate Fusion Industry Materials Other

9. 9 Rogers_NCCS_Hardware_SC07 Jaguar – Cray XT4 Jaguar Utilization by Discipline 100 OctDecFebAprJunAug Utilization (%) 90 80 70 60 50 40 30 20 10 0 Astro Biology Chemistry Climate Fusion Industry Materials Other Combustion Nuclear

10. 10 Rogers_NCCS_Hardware_SC07 Contact Jim Rogers Director of Operations National Center for Computational Sciences (865) 576-2978 jrogers@ornl.gov 10 Rogers_NCCS_Hardware_SC07