Highest performance parallel storage for HPC environments Garth Gibson CTO & Founder IDC HPC User Forum, I/O and Storage Panel April 21, 2009.

Slides:

Advertisements

Similar presentations

Issues of HPC software From the experience of TH-1A Lu Yutong NUDT.

Advertisements

Introduction Why do we need Mobile OGSI.NET? Drawbacks:

PNFS, 61 th IETF, DC1 pNFS: Requirements 61 th IETF – DC November 10, 2004.

HPC USER FORUM I/O PANEL April 2009 Roanoke, VA Panel questions: 1 response per question Limit length to 1 slide.

Introduction to DBA.

PNFS Update Sorin Faibish, EMC. pNFS Update Outline What is pNFS? pNFS Timeline Standards Status Industry Support pNFS EMC Status.

Information Technology Center Introduction to High Performance Computing at KFUPM.

Cisco and NetApp Confidential. Distributed under non-disclosure only. Name Date FlexPod Entry-level Solution FlexPod Value, Sized Right for Smaller Workloads.

Silicon Graphics, Inc. Poster Presented by: SGI Proprietary Technologies for Breakthrough Research Rosario Caltabiano North East Higher Education & Research.

A Comparative Study of Network Protocols & Interconnect for Cluster Computing Performance Evaluation of Fast Ethernet, Gigabit Ethernet and Myrinet.

Building a Cluster Support Service Implementation of the SCS Program UC Computing Services Conference Gary Jung SCS Project Manager

Peter Stefan, NIIF 29 June, 2007, Amsterdam, The Netherlands NIIF Storage Services Collaboration on Storage Services.

Accelerating Time to Results KC ZHANG Panasas Technical and Business Development Manager Leader in Parallel Storage Systems.

PolyServe Matrix Server™ Redefining the Way Servers Talk to Storage Carter George VP, Corporate Development PolyServe.

SGI Proprietary SGI Update IDC HPC User Forum September, 2008.

1 Advanced Storage Technologies for High Performance Computing Sorin, Faibish EMC NAS Senior Technologist IDC HPC User Forum, April 14-16, Norfolk, VA.

© 2008 The MathWorks, Inc. ® ® Parallel Computing with MATLAB ® Silvina Grad-Freilich Manager, Parallel Computing Marketing

Roland Dreier Technical Lead – Cisco Systems, Inc. OpenIB Maintainer Sean Hefty Software Engineer – Intel Corporation OpenIB Maintainer Yaron Haviv CTO.

IDC HPC User Forum Update APRIL 16, 2012 PANASAS PRODUCT MARKETING.

Silicon Building Blocks for Blade Server Designs accelerate your Innovation.

HPC USER FORUM I/O PANEL April 2009 Roanoke, VA Panel questions: 1 response per question Limit length to 1 slide.

© 2012 MELLANOX TECHNOLOGIES 1 The Exascale Interconnect Technology Rich Graham – Sr. Solutions Architect.

Extreme-scale computing systems – High performance computing systems Current No. 1 supercomputer Tianhe-2 at petaflops Pushing toward exa-scale computing.

Gilad Shainer, VP of Marketing Dec 2013 Interconnect Your Future.

Taking the Complexity out of Cluster Computing Vendor Update HPC User Forum Arend Dittmer Director Product Management HPC April,

Large Scale Test of a storage solution based on an Industry Standard Michael Ernst Brookhaven National Laboratory ADC Retreat Naples, Italy February 2,

Loosely Coupled Parallelism: Clusters. Context We have studied older archictures for loosely coupled parallelism, such as mesh’s, hypercubes etc, which.

4.2.1 Programming Models Technology drivers – Node count, scale of parallelism within the node – Heterogeneity – Complex memory hierarchies – Failure rates.

Storage Tank in Data Grid Shin, SangYong(syshin, #6468) IBM Grid Computing August 23, 2003.

© 2010 Voltaire Inc. HPCFS AT ORLANDO LUG 2011 BILL BOAS PATH FORWARD FOR LUSTRE COMMUNITY System Fabric Works.

Crystal Ball Panel ORNL Heterogeneous Distributed Computing Research Al Geist ORNL March 6, 2003 SOS 7.

AlphaServer UNIX Resource Consolidation.

Looking Ahead: A New PSU Research Cloud Architecture Chuck Gilbert - Systems Architect and Systems Team Lead Research CI Coordinating Committee Meeting.

1 SIAC 2000 Program. 2 SIAC 2000 at a Glance AMLunchPMDinner SunCondor MonNOWHPCGlobusClusters TuePVMMPIClustersHPVM WedCondorHPVM.

March 9, 2015 San Jose Compute Engineering Workshop.

Oct 15, PETASCALE DATA STORAGE INSTITUTE The Drive to Petascale Computing Faster computers need more data, faster. --

SoCal Infrastructure OptIPuter Southern California Network Infrastructure Philip Papadopoulos OptIPuter Co-PI University of California, San Diego Program.

Jump to first page One-gigabit Router Oskar E. Bruening and Cemal Akcaba Advisor: Prof. Agarwal.

Click to Edit Master Subtitle Style HPC USER FORUM I/O PANEL April 2009 Roanoke, VA Panel questions: 1 response per question Limit length to 1 slide.

1 Raspberry Pi HPC Testbed By Bradford W. Bazemore Georgia Southern University.

Large Scale Parallel File System and Cluster Management ICT, CAS.

PNFS BOF FAST Sorin Faibish, EMC Mike Eisler, NetApp Brent Welch, Panasas Piyush Shivam, Sun Microsystems.

WP18: High-Speed Data Recording Krzysztof Wrona, European XFEL 07 October 2011 CRISP.

Interactive Supercomputing Update IDC HPC User’s Forum, September 2008.

Forschungszentrum Karlsruhe in der Helmholtz-Gemeinschaft Implementation of a reliable and expandable on-line storage for compute clusters Jos van Wezel.

High Performance Storage Solutions April 2010 Larry Jones VP, Product Marketing.

Mellanox Connectivity Solutions for Scalable HPC Highest Performing, Most Efficient End-to-End Connectivity for Servers and Storage April 2010.

Internet2 AdvCollab Apps 1 Access Grid Vision To create virtual spaces where distributed people can work together. Challenges:

Partitioned Multistack Evironments for Exascale Systems Jack Lange Assistant Professor University of Pittsburgh.

Plumbing the Computing Platforms of Big Data Dilma Da Silva Professor & Department Head Computer Science & Engineering Texas A&M University.

Construction of Computational Segment at TSU HEPI Erekle Magradze Zurab Modebadze.

PNFS Birds-of-Feather FAST 2010: February 24 Sorin Faibish, EMC and pNFS friends.

Comprehensive Scientific Support Of Large Scale Parallel Computation David Skinner, NERSC.

Barriers to IB adoption (Storage Perspective) Ashish Batwara Software Solution Architect May 01, 2007.

Accelerating High Performance Cluster Computing Through the Reduction of File System Latency David Fellinger Chief Scientist, DDN Storage ©2015 Dartadirect.

Mellanox Connectivity Solutions for Scalable HPC Highest Performing, Most Efficient End-to-End Connectivity for Servers and Storage September 2010 Brandon.

Ian Bird CERN, 17 th July 2013 July 17, 2013

IDC HPC USER FORUM Weather & Climate PANEL September 2009 Broomfield, CO Panel questions: 1 response per question Limit length to 1 slide.

Tackling I/O Issues 1 David Race 16 March 2010.

 Cloud Computing technology basics Platform Evolution Advantages  Microsoft Windows Azure technology basics Windows Azure – A Lap around the platform.

Introduction to Data Analysis with R on HPC Texas Advanced Computing Center Feb

Presented by SciDAC-2 Petascale Data Storage Institute Philip C. Roth Computer Science and Mathematics Future Technologies Group.

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING CLOUD COMPUTING

Appro Xtreme-X Supercomputers

OCP: High Performance Computing Project

HPC USER FORUM I/O PANEL April 2009 Roanoke, VA

Fundamentals of Computer Networks

Big Data, Simulations and HPC Convergence

Presentation transcript:

Highest performance parallel storage for HPC environments Garth Gibson CTO & Founder IDC HPC User Forum, I/O and Storage Panel April 21, 2009

Slide 2 Panasas, Inc. IDC Storage Panel Question 1 With the formalization of Parallel NFS as a standard, what steps are being provided to enable this to be hosted on current (and future) platform choices? Panasas has contributed two engineers to the open source Linux code effortfor pNFS. Code has started to be included in Linux (2.6.30) and RedHat is beginning to grease the path into Fedora and RHEL. Panasas has an implementation working with the current Linux head of line code. Los Alamos National Lab has started testing with Panasas on a 128 node cluster. Ultimate testing may be done on the world’s fastest supercomputer today, LANL’s Roadrunner, which runs PanFS now. LANL also has plans to test with IBM, BlueArc and maybe with LSI.

Slide 3 Panasas, Inc. IDC Storage Panel Question 2 What are the tools available to help optimize pNFS from the application level down? Panasas collaborates with the DoE Petascale Data Storage Institute (PDSI) on its agenda to clear the path to Exascale. PDSI has built tracing tools and benchmarks for complex HPC codes that will be applied to pNFS. In the toolkit of NFS development the most important tool is frequent interoperability testing at Connectathon face-to-face engineering meetings. For debugging, Wireshark has added NFSv4.1 (pNFS) trace specific parsing rules.

Slide 4 Panasas, Inc. IDC Storage Panel Question 3 We are all facing complexity and cost issues. With IB or 10 GE (40/100 GE), where should the HPC community focus its resources? Panasas supports 1GE, 10GE, and IB connectivity. We believe 10GE is the most compelling long term, especially with data center ethernet protocol enhancements, but IB is quite effective today.

Slide 5 Panasas, Inc. IDC Storage Panel Question 4 There are too many standards, interconnects, media layers today. iSCSI/FCoIB/FCoE/FCoCEE have all been touted as the solution. Is this even relevant in the HPC arena? Is fragmentation the only choice? Panasas believes in converged networks based on Ethernet. We use iSCSI today in solutions of up to 30 GB/s. Converged and Enhanced Ethernet, or Data Center Ethernet, is an interesting optimization, but is not required. Panasas does not see FC or any of its variants to be important in HPC. It might be incidentally present inside storage boxes, although SAS should be fine for this too.

Slide 6 Panasas, Inc. IDC Storage Panel Question 5 What are the top 3 main (technical or human) issues in HPC I/O today? While high performance parallel file systems now support concurrent writing into one file system from 1000s of nodes at the same time, large numbers of codes are still written for the era in which storage was connected to only one node. Pushing all I/O through one node does not scale, but adapting and optimizing stable code bases takes many years. Enterprise-class reliability and availability has long been absent in high performance, scalable HPC systems. Too many solutions are too do-it- yourself to be reliable, well integrated, well supported AND scalable. Multiple divergent business models confuse things – is scalable storage software developed and tested by hackers in their basements, is it a loss leader cost center buried in the price of compute cluster hardware, or is it a valuable asset in itself?