1. © 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice HP Unified Cluster Portfolio with Cisco InfiniBand R. Kent Koeninger HPCD Product and Technology Marketing May 2007

2. 2HP Confidential Disclosure Agreement (CDA) Required HP Delivers Complete HPC Solutions Innovation based on standards Broadest choice of customer-focused HPC solutions Affordable, accessible, supercomputing performance

3. 3 HPC Cluster Interconnects Requirements Make it go fast Highly reliable, but usually not HA Low price −Spend the budget on compute services High scalability and throughput −Connect many servers in clusters, farms, and grids −Support efficient distributed-parallel execution High bandwidth, low latency, high message rate, low overhead High speed parallel-scalable filesystem connectivity Compatible with existing applications −Portability lowers TCO Easy to deploy, use, and upgrade −Best initial and long-term TCO

4. 4HP Confidential Disclosure Agreement (CDA) Required MPICH V1.2.5 MPI-1 Application MPI-1 (built shared) HP-MPI V2.1 MPI-1 MPI-2 HP-MPI: ISV Preferred for Performance and Transport & Object Compatibility MPICH Compatible MPI-1 Linux Itanium Linux x86 XC V2.0 HP-MPI V2.1 and later is object compatible with MPICH V1.2.5 and later Object Compatibility Across MPI Versions Multiple OS & cluster compatibility XC Clusters Object compatibility across transports Gb Ethernet

5. 5 HP Unified Cluster Portfolio Interconnects: Fast communication for fast computation UCP integrated portfolio of cluster interconnects −High performance and high scalability −Low price −Industry leading and industry standard 1 Gbps Ethernet (GbE and dual-GbE) − Inexpensive and sufficient for many HPC computational clusters InfiniBand (10 Gbps and 20 Gbps) −Enables high scalability for distributed-parallel applications −Delivers lower latency and higher packet rates for MPI, sockets, … Other interconnect solutions available by request New: UCP with Cisco InfiniBand

6. 6 Faster Interconnects  Faster Clusters High performance: bandwidth, latency & packet rate Maximum parallel application performance and scalability Portfolio of industry- standard interconnects Best price, Best performance, Fastest time to market Open, industry-standard software interfaces Application compatibility and portability HP-MPI and sockets on faster, lower-level, software interfaces Highest performance and highest parallel- programming scalability

7. 7 Low cost Ethernet interconnects −Sufficient for the majority of HPC clusters − Higher performance InfiniBand interconnects −For demanding distributed-parallel message-passing (MPI) HP Unified Cluster Portfolio High Performance Interconnects GigE 60-80MB/s, >40 μSec MPI latency InfiniBand −IB 4x DDR – speeds 2.4-2.6 GB/s −3-4 μSec MPI latency Ethernet

8. 8 Gigabit Ethernet Low price Sufficient for a majority of HPC clusters −Good for many independent processes −Also OK for low-scale message-passing parallel-distributed codes Good access to storage −100 MB/s per server −Sufficient for low core counts (too slow for 16 core servers?) Dual and quad GbE for Enterprise uses −InfiniBand can often under price and outperform bonded GbE 10GbE for backbone (switch to switch) use Ethernet

9. 9HP Confidential Disclosure Agreement (CDA) Required Why Interconnects Matter? 2 Nodes 4 Nodes8 Nodes Fluent Perf Study – IB versus GigE Almost linear speedup with IB 3.6M call model on 1 to 16 cores GigE does not scale beyond small clusters

10. 10HP Confidential Disclosure Agreement (CDA) Required What is InfiniBand? Industry standard switched fabric Performance −High bandwidth 20Gb/s each-direction w/ 4X DDR (double data rate link) Future, 40Gb/s QDR is expected in 2008 timeframe −Very low latency 3 ~ 4usec MPI Ping-pong w/ Mellanox technology −Very low CPU usage during message-passing Enables computation and message-passing overlap Scalability −Thousands of nodes Ease of clustering −Self-discovery of nodes Plug and play

11. HP Confidential Disclosure Agreement (CDA) Required Slide 11 InfiniBand Transport Offload Eliminated the systems bottlenecks with RDMA −Kernel bypass −Handling protocol transport in the adapter −Zero-copy operations

12. 12 High bandwidth, low latency Ethernet −Ethernet compatibility with HPC performance 10GbE expected to sell well in Enterprise markets −Available 2007, ramping 2008, stride 2009 −HP BladeSystem c-Class 10GbE switching 40 Gbps per switch module −4 external 10 GbE ports and 16 internal 1GbE ports in switch module 100 Gbps per switch module −10 external and 10 internal 10GbE ports in switch module −Two modules for 10 GbE per blade across 16 blades 10GbE not expected to ramp in HPC markets −IB has lower price, higher bandwidth, better latency, faster message rates, higher cluster scalability −Growing demand for IB to10 GbE gateways (6 to 9 months away) 10 Gb Ethernet with RDMA (RNICs) 10GigE RDMA RNICs shipping mid 2007 Near 1 GB/s with under 10 μSec MPI latency

13. 13 Cisco InfiniBand in HP Cluster Platform and BladeSystem c-Class Clusters HP branded Mellanox 4X DDR PCIe HCAs (DL and BL) with Cisco Drivers for c-Class Cisco SFS 7000D Switch Series 24, 144 & 288 port DDR HP 4X DDR single-port c-Class mezzanine card HP DDR 24-port c-Class switch module BladeSystem c-Class Cisco IB drivers & FM SFS7024D SFS7012D SFS7000D Cisco Fabric Manager

14. 14 Cisco InfiniBand Software Stacks for BladeSystem c-Class Clusters (Cisco drivers not sold by HP on rack-mount servers) OFED (Open Fabrics Enterprise Edition) Compatible with XC 3.2 and HP SFS V2.2-1 Cisco InfiniBand drivers for BladeSystem c-Class clusters −3 versions: Commercial Linux, OFED Linux & Windows Windows not yet available −Recommended for c-Class customers who prefer Cisco’s networking ecosystem

15. 15HP Confidential Disclosure Agreement (CDA) Required HCAsHost Drivers Fabric Managers SwitchesConverge d Fabrics Rack mount: Mellanox 4X-SDR or 4X-DDR PCIe or PCI-X 1 or 2 ports Memory options General: OFED 1.1 stack In XC 3.2 and HP SFS V2.2-1 c-Class: Mellanox DDR mezzanine (single port) c-Class Cisco proprietary * & OFED drivers (soon for Windows) C-Class Cisco Fabric Manager (switch or host based) Cisco: DDR 24, 144, 288 Cisco 3012 (Reference sale from Cisco, not from HP) c-Class: Mellanox: DDR 24 (2:1) (8 external ports, 16 internal ports) HP SFS IB Storage Direct-connect IB-SAN Storage (UCP/CP support TBD) UCP Linux Cisco-InfiniBand CY2Q2007 (May) Options to sell and ship with CP support * In CP but not supported by XC or HP SFS

16. 16HP Confidential Disclosure Agreement (CDA) Required Cisco HPC InfiniBand Solution Building Blocks Gateway Modules - IB to Ethernet -IB to Fibre Channel (Cisco reference parts: not parts from HP) Embedded System and Fabric management Cisco InfiniBand DDR Server Fabric Switches Linux Drivers for c-Class Host Channel Adapter  MPI  IPoIB  SDP  … Cisco Commercial and OFED (Windows TBD) Embedded and Hosted Subnet Manager CiscoWorks & Virtualization Software Packages

17. Cisco SFS 7000D 24 dual-speed InfiniBand 4X ports −20-Gbps double data rate (DDR) −10-Gbps single data rate (SDR) Non-blocking 480-Gbps cross-sectional bandwidth Port-to-Port latency less than 200 nanoseconds Embedded subnet manager Std Cisco CLI, Web, and Java-based systems management options Powered ports for flexible copper and optical interfaces Cisco specified and exclusive belly-to-belly IB connectors; DDR signals reach user ports without traversing mezzanine connector.

18. 18HP Confidential Disclosure Agreement (CDA) Required Cisco Large Switch DDR Family SFS7012D Port Module Options Chassis Type Subnet Manager High Availability Best Use 14U Modular 24 Side by Side Slots 12 by 4X LIMs Powered Interfaces External 7U Modular Max 4X ports 288 ports144 ports 12 Side by Side Slots 12 by 4X LIMs Powered Interfaces External Link Speed DDR Redundant Power/Cooling Redundant Management Hot Swappable FRUs Redundant Power/Cooling Redundant Management Hot Swappable FRUs  97-144 node clusters  145-288 node clusters  Core switch for 1,536+ node clusters SFS7024D

19. © 2006 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice Cisco InfiniBand in c-Class BladeSystem Clusters

20. 20HP Confidential Disclosure Agreement (CDA) Required BladeSystem - Cisco Infiniband Solution HP c-Class BladeSystem InfiniBand Mellanox HCA and Switch Module Cisco InfiniBand Host Based Software Drivers Cisco SFS 7000D InfiniBand Switches & Fabric Managers

21. 21HP Confidential Disclosure Agreement (CDA) Required 16 16 BL460c with one HCA each DDR IB Switch Module c7000 16 c7000 Up to 32 nodes cluster configuration (2 switch hops) 8 8 With OFED drivers: Requires at least one SFS7000D (24 port) switch to run the Cisco Fabric Manager Small configuration example with c-Class Note: other Ethernet networks are not drawn in this diagram

22. 22HP Confidential Disclosure Agreement (CDA) Required Single rack example with c-Class 16 c7000 16 c7000 12 3 4 4 4 Up to 48 nodes cluster configuration Subnet manager runs on 24-port switch Fabric redundancy Max switch hops: 3 SFS7000D 24-port DDR IB switch 16 Note: Other Ethernet networks are not drawn in this diagram. 4 4 4 SFS700D 24-port DDR IB switch Leaf-level SW Spine-level SW

23. 23HP Confidential Disclosure Agreement (CDA) Required Multi-rack configuration example with c-Class Use 24-port SFS700D switches for up to 384 IB-port configurations Use larger SFS7012D and SFS7024D switches for larger c-Class clusters 256 nodes cluster configuration w/ 8 24-port switches S ubnet manager runs on switch Fabric redundancy Max switch hops: 3 16 1 c7000 16 1 c7000 16 1 c7000 1 1 1 1216 … … Note: other Ethernet networks are not drawn in this diagram 24-port DDR IB SW * * *

24. 24HP Confidential Disclosure Agreement (CDA) Required Scaling clusters with larger switch 16 c7000 16 c7000 12 … 32 … 8 88 8x36=288 ports 512-node cluster configuration with single 288-port switch (up to 5 switch hops) SFS7024D 288 port switch 16 Requires at least one SFS7000D (24 port) switch to run the Cisco Fabric Manager

25. 25 Higher bandwidth, lower latency Ethernet −Ethernet compatibility with HPC performance for MPI InfiniBand performance improvements −Continuing the lead in performance: 20 Gbps  40 Gbps Expected HP Unified Cluster Portfolio Interconnect Improvements (not necessarily yet plan of record) 10GigE RDMA in mid 2007 Near 1 GB/s with under 10 μSec MPI latency InfiniBand −Lower latency HCAs in 2007 (under 2 μSec s ) −PCI Express 2.0 (DDR) in 2008 −Quad data rate (QDR) in 2008/2009 (over 3 GB/s) top-level switches (288 ports) node-level switches (24 ports) Connects to 12 nodes

26. © 2006 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 26 Cisco IB Value

27. HP Confidential Disclosure Agreement (CDA) Required 27 Cisco HPC Market Leadership-Top 500 List  7 of Top 10 systems use Cisco networking  Top 3 largest InfiniBand Clusters running Cisco InfiniBand**  InfiniBand is growing with 22% attachment  InfiniBand is based on open standards w/Cisco as active contributor  Ethernet is predominant interconnect with 60% attachment  Myrinet, Quadrics are dropping down based on proprietary technologies *Growth numbers shown here reflect changes in Top 500 systems, they don’t necessarily reflect new purchases ** Size of cluster is measured by Number of Nodes and not Teraflops, Sandia National Labs #6, Maui HPCC #11 & TACC #12 Nov 2006 Top 500 List  359 Clusters in Top 500 211 Gigabit Ethernet 79 InfiniBand 51 Myrinet 13 Quadrics

28. HP Confidential Disclosure Agreement (CDA) Required 28 Cisco InfiniBand Differentiators  Most Complete DDR Switching Line - Cisco is shipping fixed and modular DDR switches with best in class bit error rates (10 -15 )  Scalable InfiniBand Subnet Manager - Proven scalability of 4600 nodes, Sandia Labs - High availability across multiple SM’s with full database synchronization - Rapid fabric discovery and fabric “bring up”.. Less than 60 seconds for 4000 nodes - Optimized routing for SDR/DDR mixed fabrics  InfiniBand Host Drivers  Enterprise Class Mgmt and Security SNMP V3 for capturing chassis failures, performance counters, fabric topology, logging over SNMP SNMP offers integration CiscoWorks and HP OpenView! Imaging upgrading FTP, TFTP, SCP Radius and TACACS+, integrated with Cisco ACS  High Performance I/O - High Performance and Highly Available Ethernet Gateway for Client, NAS & Parallel File System Access - SRP to Fibre Channel Gateway for SAN Access - Remote server boot for virtualization

29. HP Confidential Disclosure Agreement (CDA) Required 29 Cisco SFS InfiniBand Switch Differentiators CategoryDifferentiation Experience and Expertise  Proven solution in the largest HPC (5000+ servers) and enterprise clusters in production  5 Financials in Production in Bulge Bracket; 10+ pilot engagements in Financial Services  From Cisco, the leader in Market Data Implementations Enterprise-Class Product Line  Robust, enterprise class end-to-end solution including SAN and NAS interconnects (JPMC – 0.99999 reliability at the Compute Backbone)  Fastest failover and recovery around faults (sub millisecond reroutes around link failures)  Broadest portfolio of IB products (IB Switches, Blade-Switches, GWs, HCAs, Subnet Manager, Linux and Windows Drivers)  Standards based; leading industry efforts to create interoperable solutions (OpenFabrics, Open MPI, HP-MPI) Comprehensive Ecosystem  Comprehensive vertical focused solutions (FSI, Manufacturing, CAE, Oil and Gas, Storage)  Key relationships up and down the value chain (Intel/AMD, RHAT/SuSE, Wombat/TIBCO, Oracle…)  Broadest industry interoperability matrix - Processor (Intel, AMD, IBM), OS, ISV qualifications World-class Support  24 x 7 x 365 global support; Depots in 122 countries  Best practices (tools and documentation) and services for installation, fabric validation  Global training, scalable advanced services, robust escalations From Cisco – Leading Interconnect Provider  Best value solutions in the industry (Yankee Group – 5/18/06)  Leading provider of networking solutions; Broadest portfolio of products  Success based on intense customer satisfaction focus  Robust product lifecycle management, deep financial backing (Investment protection)

30. HP Confidential Disclosure Agreement (CDA) Required 30 Linux Commercial Stack - Differentiators CategoryDifferentiator GeneralStack has been used for production for 24+ months; Proven functionality, stability and scalability IPoIB-Robust IPoIB with no single point of failure between servers or with EXGW -Industry leading EXGW (performance, functionality and HA) -Proven scalability to 1200 servers in demanding enterprise environment SRP-Robust SRP with no single point of failure (withstands 50 sec cable interruption) -Transparent model abstracts Ib fabric enabling all SCSI level utilities to work (ex: multipathing software – qualification necessary; but competitors have fundamental architecture issues getting multipathing to work) MVAPICH-Proven scalability to 4700 servers at Sandia SDP-At par with competitors; Implementing zero-copy and asynchronous IO to gain advantage DAL-Implementing UDP unicast & multicast accelerator (for FSI) RemoteBoot- Only company to support SAN and PXE in production -Proven scalability to 256 nodes -RemoteBoot-SAN

31. HP Confidential Disclosure Agreement (CDA) Required 31 Cisco Components of OFED1.1 ComponentSupportedPurposeIncluded in Cisco Package Maintainer Core IBYesCore IB components (SMA, CMA etc.)YesCisco mTHCAYesHCA Driver for Mellanox HCAYesCisco iPath*NoHCA Driver for Pathscale HCANoPathscale IPoIBYesTransport IP over InfiniBandYesCisco SRP Initiator*YesBlock level access from IB attached serverYesCisco OSU MVAPICHYesOSU MVAPICH (MPICH over VAPI)YesMellanox Open MPI*YesOpen MPI support over InfiniBandYes*Cisco uDAPL*YesuDAPL SupportYesIntel iSER Initiator*NoiSCSI over RDMA InitiatorNoVoltaire OpenSM*NoSubnet ManagerNoMellanox SDP*YesSockets Direct ProtocolYesMellanox Diagnostic ToolsYes (Some)Diagnostics to verify cluster healthYesMellanox Performance TestsYesMeasure performance of various protocolsYesMellanox Cisco is primary SQA tester of OFED

32. HP Confidential Disclosure Agreement (CDA) Required 32 Thank You!