NICS Update Bruce Loftis 16 December 2009

National Institute for Computational Sciences University of Tennessee and ORNL partnership  NICS is the 2 nd NSF Track 2 center –Builds on strengths of UT and ORNL  NICS operates the first academic petascale supercomputer in the world  Staff of 25 2Managed by UT-Battelle for the Department of Energy

NICS Timeline 3 Track-2B Award announced for University of Tennessee September 2007 Cray XT3 Production 7K cores, 40 TFJune 2008 Cray XT4 Production 18K cores,166 TFAugust 2008 Cray XT5 Production 65K cores, 600 TF 2.3 GHz, quad-core Barcelona February 2009 Cray XT5 Istanbul upgrade, Production ~100,000 cores, 2.6 GHz, 1 PF November 2009 (Actual: Oct 5, 2009) Delivered over 305M hours to researchers!

Upgrade to Istanbul Processors September 2009 BarcelonaIstanbulImpact Clock Speed2.3 GHz2.6 GHz Core Counts8 per node12 per node  70% more FLOPS Memory Bandwidth 18.1 GB/s/node > 21.2 GB/s/node  17% more memory bandwidth Level 3 Cache4 MB/node12 MB/node  3x more cache Memory Upgrademixed16 GB / compute node

COLA NICS JAMSTEC U of Tokyo ECMWF COLACenter for Ocean-Land-Atmosphere Studies, Calverton, MD, USA ECMWFEuropean Center for Medium-Range Weather Forecasts, Reading, England JAMSTECJapan Agency for Marine-Earth Science and Technology, Tokyo, Japan UTokyoUniversity of Tokyo, Japan NICSNational Institute for Computational Sciences, University of Tennessee, Knoxville, TN USA An International, Dedicated High-End Computing Project to Revolutionize Climate Modeling

NICAM Satelite Observation Model Simulation The only global atmospheric model capable of resolving clouds

User Survey – Big Results –Requirements for computing capability and archival storage will continue to grow  15 respondents expect to store as much as 1 Pbyte annually in the next 5 years –Significant interest in exploring GPGPU’s to increase computing capability –Concern about visualization and analysis of output data  moving large datasets home or elsewhere is a challenge  remote visualization is appealing

Keeneland – An NSF-Funded Partnership to Enable Large-Scale Computational Science on Heterogeneous Architectures  Track-2D: Experimental HPC System of Innovative Design – Large GPU cluster –Initial delivery Fermi system – Spring 2010 –Full scale system – Spring 2012  Partners are: Georgia Tech, NICS, U of Tennessee, Oak Ridge, NVIDIA, HP  Software tools, application development  TeraGrid Resource Provider  Operations, User Support at NICS  Education, Outreach, Training for scientists, students, industry 9

Sean Ahern, PI Partnership between UT, ORNL, LBNL, U of Wisconsin and NCSA Purpose: provide TeraGrid and XD users with remote and shared memory resources for exploring, analyzing, and visualizing large- scale data Central hardware: Nautilus, a shared memory SGI UltraViolet system with: ‣ 1024 Intel Nehalem EX cores ‣ 4 TB of shared memory ‣ 16 GPUs ‣ ~1 PB filesystem ‣ TeraGrid and Kraken connectivity Provide wide range of software tools for data analysis, visualization, and workflow management

Timeline for Deployment

NICS Education, Outreach, and Training  “Introduction to Petascale Computing” workshop for the 2009 Richard Tapia Celebration of Diversity in Computing  Co-sponsored and organized workshop for NIMBioS Center at UT-Knoxville.  Outreach to student chapter of the Society of Women Engineers at UT- Knoxville.  Two million CPU hours committed to Institute for Computing in the Humanities, Arts and Social Sciences.  Participated in student and teacher programs at TG ‘09 and SC09.  Virtual School for Computational Science and Engineering: two Summer Schools hosted during summer of 2009; hosting three week-long schools in  Co-sponsored Hex-Core Cray XT5 workshops in December 2009 (ORNL) and February 2010 (UC-Berkeley)  TeraGrid/Blue Waters technical workshop, Austin, March 2010  TeraGrid Introductory training via ReadyTalk, quarterly, online.  Introductory workshops for UTK users and undergraduate classes.

Thank You !!