1. High performance and technical computing in Microsoft Computer and Information Sciences Life Sciences Multidisciplinary Research Math and Physical Science Earth Sciences Fabrizio Gagliardi Director, Technical Computing, Microsoft EMEA&LATAM Social Sciences New Materials, Technologies and Processes

2. 2 Outline A few personal remarks Defining High Performance Computing (HPC) Industry and Market Trends Technology Challenges Microsoft technical & scientific computing Windows Compute Cluster Server 2003 Summary

3. 3 A few personal remarks Been here before as initiator and leader of many related Grid projects, EGEE and EGEE-II above all 30 years at CERN developing HPC solutions for science Pioneered Expert Systems, AI, MPP systems and in the last 7 years Grid computin (founding member of GGF) Joined Microsoft on 1/11/2005 after EGEE_04 in Pisa: Director for technical computing in Europe, Middle East, Africa and Latino America

4. Enabling Grids for E-sciencE 4 Related EU projects ISS e G

5. 5 High Performance Computing Cutting edge problems in science, engineering and business always require capabilities beyond those of standalone computers Market pressures demand accelerated innovation cycle, overall cost reduction and thorough outcome modeling Aircraft design utilizing composite materials Vehicle fuel efficiency and safety improvements Simulations of enzyme catalysis, protein folding Targeted material and drug design Simulation of nanoscale electronic devices Financial portfolio risk modeling Digital content creation and enhancement Supply chain modeling and optimization Long term climate projections Volume economics of industry standard hardware and commercial software applications are rapidly bringing HPC capabilities to broader number of users

6. 6 Defining High Performance Computing Systems MPPs, Vector, NUMA Dedicated HPC Clusters Resource Scavenging Goal Absolute Performance Price/Perf, SLA Harness unused cycles Targets Large Scale SMP Clusters Underutilized Desktops & Servers Coupling Extreme Tightly, Loosely Loosely Comm Shared Memory MPI, noneProprietary Network Bus, backplane Myrinet, Infiniband, GigE TCP/IP – LAN HPC Role in Science Different Platforms for Achieving Results Computational Modeling Sensors Interpretation & Insight Persist Data Mining & Algorithms HPC Definition: Using compute resources to solve computationally intensive problems

7. 7 HPC Market Trends

8. 8 Top 500 Supercomputer Trends Industry usage rising Clusters over 50% x86 is leading GigE is gaining

9. 9 Supercomputing Goes Personal 199119982005 System Cray Y-MP C916Sun HPC10000Small Form Factor PCs Architectur e 16 x Vector 4GB, Bus 24 x 333MHz Ultra- SPARCII, 24GB, SBus 4 x 2.2GHz Athlon64 4GB, GigE OS UNICOSSolaris 2.5.1Windows Server 2003 SP1 GFlops~10 Top500 # 1500N/A Price $40,000,000$1,000,000 (40x drop) < $4,000 (250x drop) Customers Government LabsLarge EnterprisesEvery Engineer & Scientist Application s Classified, Climate, Physics Research Manufacturing, Energy, Finance, Telecom Bioinformatics, Materials Sciences, Digital Media

10. 10 HPC Scenarios Evolving ScenariosKey Factors Batch Computing on SupercomputersCompute cycles are scarce and require careful partitioning and allocation Cluster systems administration major challenge Applications split into UI and compute parts Interactive Computing on Departmental Clusters Compute cycles are cheap Interactive applications integrate UI/compute parts Emergence of turnkey personal clusters Complex Workflow Spanning ApplicationsCompute and data resources are diffused throughout the enterprise Distributed application, systems and data management is the key source of complexity Multiple applications are organized into complex workflows and data pipelines Interactive Computation and Visualization Manual, batch execution IT Mgr SQL

11. 11 Technology challenges Moore’s law continues but power consumption and heat dissipation are reaching their limits Memory and data access gap widen Applications become more data intensive

12. 12 Data Access Time CPU cycle: 2 GHZ (10 -9 ) Memory access: 60 ns (10 -9 ) Disk access: 10 ms (10 -3 ) Translating the referential by (2 GHZ): CPU cycle ~1 second Memory access ~120 seconds (~2 minutes) Disk access ~20 000 000 seconds (~6 months)

13. 13 The Future: Supercomputing on a Chip IBM Cell processor 256 Gflops today 4 node personal cluster => 1 Tflops 32 node personal cluster => Top100 MS Xbox 3 custom PowerPCs + ATI graphics processor 1 Tflops today $300 8 node personal cluster => “Top100” for $2500 (ignoring all that you don’t get for $300) Intel many-core chips “100’s of cores on a chip in 2015” (Justin Rattner, Intel: http://www.hpcwire.com/hpc/629783.html ) “4 cores”/Tflop => 25 Tflops/chip

14. 14 The Microsoft project in Barcelona Microsoft is interested in helping computer scientists to develop new computing architectures with a high level of parallelism Mateo Valero and his BSC centre in Barcelona are leaders in this field in Europe Microsoft will invest considerable resources over the next two years at BSC to research and develop an entirely new parallel computing ecosystem http://www.hpcwire.com/hpc/633342.html

15. 15 Microsoft Technical Computing: Radical Computing Research in potential breakthrough technologies Advanced Computing for Science and Engineering Application of new algorithms, tools and technologies to scientific and engineering problems High Performance Computing Application of high performance clusters and database technologies to industrial applications

16. 16 Microsoft and HPC: Microsoft has announced at the last SC’05 conference major investments and products in the HPC area Products will start shipping soon, some are already available as beta from the Microsoft HPC site Today HPC is becoming commodity computing of the future

17. 17 The Continuing Trend Towards Decentralized, Networked Resources Grids of personal & departmental clusters Personal workstations & departmental servers Minicomputers Mainframes

18. 18 MS HPC road map Computer Cluster Solution (CCS) V1: Beowulf-style compute cluster seamlessly integrated into a Windows-based infrastructure, including security infrastructure (Kerberos and Active Directory); user jobs run under the Windows user credentials. ISVs support for most common software applications. Job scheduler accessible via command-line, COM and via a published Web services protocol. Can be customized via sys admin-defined admission and release filters that run in the scheduler when a job is submitted and when it becomes scheduled as ready-to-run. In V2 the scheduler will be made even more extensible. Performance will be comparable to Linux. MPICH (incl. MPICH-2) supported (open-source version from ANL).

19. 19 MS HPC road map CCS areas of focus for V2: Extend to forests of clusters and meta-schedulers Storage and parallel I/O issues, as well as possibly simple workflow support Development of new the tools

20. 20 Windows Compute Cluster Server 2003 Deliver the easiest to deploy and most cost effective solution for solving scaled-out business, engineering and scientific computational problems. Mission: Deliver the easiest to deploy and most cost effective solution for solving scaled-out business, engineering and scientific computational problems. Windows Server 2003, Compute Cluster Edition Compute Cluster Pack += Microsoft Windows Compute Cluster Server 2003 Support for high performance hardware (x64bit architecture)Support for high performance hardware (x64bit architecture) RDMAsupport for high performance interconnects (Gigabit Ethernet, Infiniband, Myrinet, and others)RDMA support for high performance interconnects (Gigabit Ethernet, Infiniband, Myrinet, and others) Support for Industry Standards MPI2Support for Industry Standards MPI2 Integrated Job SchedulerIntegrated Job Scheduler Cluster Resource Management ToolsCluster Resource Management Tools Integrated Solution out- of-the-boxIntegrated Solution out- of-the-box Leverages investment in Windows administration and toolsLeverages investment in Windows administration and tools Makes cluster operation easy and secure as a single systemMakes cluster operation easy and secure as a single system

21. 21 Key Features Node Deployment and Administration Compute nodes automatically imaged and added to cluster UI and command line-based n ode management To Do List to configure head node Integration with existing Windows and management infrastructure Integrates with Active Directory, Windows security technologies, management, and deployment tools Extensible job scheduler 3 rd party extensibility at job submission and/or job assignment Examples: admission policies and license verification Submit jobs from command line, UI, or directly from applications Simple management, similar to print queue management Secure MPI User credentials secured in job scheduler and compute nodes Standardized MPI stack Microsoft provided stack reduces application/MPI incompatibility issues

22. 22 HPC Innovation Centers Center Cornell Theory Center Ithaca, NY USA University of Tennessee Knoxville, TN USA TACC – University of Texas Austin, TX USA University of Virginia Charlottesville, VA USA University of Utah Salt Lake City, UT USA Tokyo Institute of Technology Tokyo, Japan HLRS – University of Stuttgart Stuttgart, Germany Southampton University Southampton, UK Shanghai Jiao Tong University Shanghai, PRC Nizhni Novgorod University Nizhni Novgorod, Russia Cornell Theory Center Ithaca, NY USA University of Tennessee Knoxville, TN USA University of Virginia Charlottesville, VA USA University of Utah Salt Lake City, UT USA TACC – University of Texas Austin, TX USA Southampton University Southampton, UK HLRS – University of Stuttgart Stuttgart, Germany Shanghai Jiao Tong University Shanghai, PRC Tokyo Institute of Technology Tokyo, Japan Nizhni Novgorod University Nizhni Novgorod, Russia Institutes for High Performance Computing

23. 23 CCS Summary Windows Compute Cluster Server 2003 removes administrative barriers preventing broad adoption of HPC solutions Familiar environment and integration with standard tools Convenient job scheduler Parallel debugging capabilities and full support of MPI standards Windows Compute Cluster Server 2003 makes HPC more accessible to scientists, engineers, and businesses in a Microsoft environment

24. 24 More information Microsoft HPC Web site http://www.microsoft.com/hpc/ Windows Server x64 information http://www.microsoft.com/64bit/ http://www.microsoft.com/x64/ Windows Server System information http://www.microsoft.com/wss/ Get the Facts Web site http://www.microsoft.com/getthefacts

25. 25 Conclusions Industry is moving HPC to commodity Microsoft is world leader in commodity computing and will play a major role in scientific and technical computing solutions Key figures in scientific computing such as Burton Smith, Tony Hey have recently joined the company in senior strategic positions You will hear from us more and more frequently in the near future Thanks for the invitation and the opportunity to talk with you in such a nice and friendly venue