Tackling I/O Issues www.openfabrics.org 1 David Race 16 March 2010.

Slides:



Advertisements
Similar presentations
Tivoli SANergy. SANs are Powerful, but... Most SANs today offer limited value One system, multiple storage devices Multiple systems, isolated zones of.
Advertisements

IBM Software Group ® Integrated Server and Virtual Storage Management an IT Optimization Infrastructure Solution from IBM Small and Medium Business Software.
Technology Drivers Traditional HPC application drivers – OS noise, resource monitoring and management, memory footprint – Complexity of resources to be.
SSRS 2008 Architecture Improvements Scale-out SSRS 2008 Report Engine Scalability Improvements.
High Performance Cluster Computing Architectures and Systems Hai Jin Internet and Cluster Computing Center.
Priority Research Direction (I/O Models, Abstractions and Software) Key challenges What will you do to address the challenges? – Develop newer I/O models.
IDC HPC User Forum Conference Appro Product Update Anthony Kenisky, VP of Sales.
2 June 2015 © Enterprise Storage Group, Inc. 1 The Case for File Server Consolidation using NAS Nancy Marrone Senior Analyst The Enterprise Storage Group,
Copyright 2009 FUJITSU TECHNOLOGY SOLUTIONS PRIMERGY Servers and Windows Server® 2008 R2 Benefit from an efficient, high performance and flexible platform.
SAN DIEGO SUPERCOMPUTER CENTER at the UNIVERSITY OF CALIFORNIA; SAN DIEGO IEEE Symposium of Massive Storage Systems, May 3-5, 2010 Data-Intensive Solutions.
Silicon Graphics, Inc. Poster Presented by: SGI Proprietary Technologies for Breakthrough Research Rosario Caltabiano North East Higher Education & Research.
1 Exploring Data Reliability Tradeoffs in Replicated Storage Systems NetSysLab The University of British Columbia Abdullah Gharaibeh Matei Ripeanu.
© 2009 IBM Corporation Statements of IBM future plans and directions are provided for information purposes only. Plans and direction are subject to change.
RAID-x: A New Distributed Disk Array for I/O-Centric Cluster Computing Kai Hwang, Hai Jin, and Roy Ho.
Grid Computing Veronique Anxolabehere Senior Director of Product Marketing Mike Margulies Senior Director, Grid Platform Solutions.
Data Deduplication in Virtualized Environments Marc Crespi, ExaGrid Systems
Distributed Systems Early Examples. Projects NOW – a Network Of Workstations University of California, Berkely Terminated about 1997 after demonstrating.
SANPoint Foundation Suite HA Robert Soderbery Sr. Director, Product Management VERITAS Software Corporation.
Word Wide Cache Distributed Caching for the Distributed Enterprise.
4.x Performance Technology drivers – Exascale systems will consist of complex configurations with a huge number of potentially heterogeneous components.
1 Advanced Storage Technologies for High Performance Computing Sorin, Faibish EMC NAS Senior Technologist IDC HPC User Forum, April 14-16, Norfolk, VA.
Bring Consolidation Into Focus The Value of Compaq AlphaServer and Storage Consolidation Solutions Joseph Batista Director Enterprise & Internet Initiatives.
Min Xu1, Yunfeng Zhu2, Patrick P. C. Lee1, Yinlong Xu2
Principles of Scalable HPC System Design March 6, 2012 Sue Kelly Sandia National Laboratories Abstract: Sandia National.
Performance Concepts Mark A. Magumba. Introduction Research done on 1058 correspondents in 2006 found that 75% OF them would not return to a website that.
CLUSTER COMPUTING STIMI K.O. ROLL NO:53 MCA B-5. INTRODUCTION  A computer cluster is a group of tightly coupled computers that work together closely.
Introduction to Cloud Computing
PACK: Prediction-Based Cloud Bandwidth and Cost Reduction System
The Red Storm High Performance Computer March 19, 2008 Sue Kelly Sandia National Laboratories Abstract: Sandia National.
High Performance Computing Processors Felix Noble Mirayma V. Rodriguez Agnes Velez Electric and Computer Engineer Department August 25, 2004.
Taking the Complexity out of Cluster Computing Vendor Update HPC User Forum Arend Dittmer Director Product Management HPC April,
Large Scale Test of a storage solution based on an Industry Standard Michael Ernst Brookhaven National Laboratory ADC Retreat Naples, Italy February 2,
AlphaServer UNIX Resource Consolidation.
March 9, 2015 San Jose Compute Engineering Workshop.
Kiew-Hong Chua a.k.a Francis Computer Network Presentation 12/5/00.
Headline in Arial Bold 30pt HPC User Forum, April 2008 John Hesterberg HPC OS Directions and Requirements.
Large Scale Parallel File System and Cluster Management ICT, CAS.
Copyright ©2003 Digitask Consultants Inc., All rights reserved Cluster Concepts Digitask Seminar November 29, 1999 Digitask Consultants, Inc.
CLUSTER COMPUTING TECHNOLOGY BY-1.SACHIN YADAV 2.MADHAV SHINDE SECTION-3.
Server Performance, Scaling, Reliability and Configuration Norman White.
11 CLUSTERING AND AVAILABILITY Chapter 11. Chapter 11: CLUSTERING AND AVAILABILITY2 OVERVIEW  Describe the clustering capabilities of Microsoft Windows.
USATLAS dCache System and Service Challenge at BNL Zhenping (Jane) Liu RHIC/ATLAS Computing Facility, Physics Department Brookhaven National Lab 10/13/2005.
VMware vSphere Configuration and Management v6
High Performance Storage Solutions April 2010 Larry Jones VP, Product Marketing.
Data Management for Decision Support Session-4 Prof. Bharat Bhasker.
Full and Para Virtualization
COMP381 by M. Hamdi 1 Clusters: Networks of WS/PC.
Axis AI Solves Challenges of Complex Data Extraction and Document Classification through Advanced Natural Language Processing and Machine Learning MICROSOFT.
Parallel IO for Cluster Computing Tran, Van Hoai.
1 5/4/05 Fermilab Mass Storage Enstore, dCache and SRM Michael Zalokar Fermilab.
The Google File System Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung Presenter: Chao-Han Tsai (Some slides adapted from the Google’s series lectures)
Unit 2 VIRTUALISATION. Unit 2 - Syllabus Basics of Virtualization Types of Virtualization Implementation Levels of Virtualization Virtualization Structures.
Red Hat Enterprise Linux Presenter name Title, Red Hat Date.
Chapter 16 Client/Server Computing Dave Bremer Otago Polytechnic, N.Z. ©2008, Prentice Hall Operating Systems: Internals and Design Principles, 6/E William.
Lecture 13 Parallel Processing. 2 What is Parallel Computing? Traditionally software has been written for serial computation. Parallel computing is the.
Univa Grid Engine Makes Work Management Automatic and Efficient, Accelerates Deployment of Cloud Services with Power of Microsoft Azure MICROSOFT AZURE.
Clouds , Grids and Clusters
Business Critical Application Platform
VirtualGL.
CLUSTER COMPUTING Presented By, Navaneeth.C.Mouly 1AY05IS037
Appro Xtreme-X Supercomputers
HPE Persistent Memory Microsoft Ignite 2017
Business Critical Application Platform
Scalable SoftNAS Cloud Protects Customers’ Mission-Critical Data in the Cloud with a Highly Available, Flexible Solution for Microsoft Azure MICROSOFT.
Quasardb Is a Fast, Reliable, and Highly Scalable Application Database, Built on Microsoft Azure and Designed Not to Buckle Under Demand MICROSOFT AZURE.
Database System Architectures
How Dell, SAP and SUSE Deliver Value Quickly
Nolan Leake Co-Founder, Cumulus Networks Paul Speciale
Presentation transcript:

Tackling I/O Issues 1 David Race 16 March 2010

Agenda The Challenge Today Customer Needs Solving Customer Needs The Appro Approach The Benefits Solution Summary 2

The issue today … 3 Supercomputers based on cluster architecture are composed of highly scalable compute nodes, fast interconnects, operating system, several programming and software tools with massive storage for very large scientific data processing and visualization. Moore’s Law in processors is not translating to storage speed. There is enough volume, but not speed. The cluster processors are in a wait state while this data is written to disk. This results in as much as 20% loss of productivity that could be used by an application. I/O bottleneck!

The Reason 4 Reduced utilization of compute cycles of the processors impacts application and database reliability and performance Data Performance Gap Registers (1 cycle) Cache (10 cycles) Memory (100 cycles) Storage (10,000 cycles) Performance Gap Current solutions don’t leverage Moore’s Law to provide ongoing bandwidth improvements 

User Needs 5 Speed and accuracy in data usage are critical Well recognized and acute I/O issues in many application areas Under budget constraints and grant limitations Has physical space and power limitations Strong supporter of standards Huge appetite for processing power Uses large data sets for modeling and simulation “Time to results” is critical Under data center limitations where use of industry-accepted hardware, software, and interface protocol standards are required Users require massive compute and data-handling capabilities to conduct their seismic and data modeling analysis.

The Appro Approach 6 The goal: Extract the maximum amount of available compute processing by reducing or eliminating the I/O bottleneck to significantly boost HPC end-user system application performance. Offer a standards-based integrated HPC architecture optimized for performance, reliability, and scalability that is non disruptive to the HPC end-user. Next Generation Solution: Appro’s next generation supercomputer solution combines a robust Storage File System and an I/O Engine software technology layer that provides an innovative way to solve the I/O bottleneck and deliver sustained application performance.

Solving Needs Computation improvements through superior I/O –Improve File Server Usage –I/O Wait-time reduction –I/O Hiding –Improved Scalability and Reliability Application programming improvements –Huge Shared Memory that spans in multiple I/O channels for 100+TB of reliable cache –Reliable Memory Paradigms –Scalable ISV environments Benefits –Size (Terabyte) and speed (GB/sec) is sized based on cache and storage –100% data reliability –No client modifications 7

The Appro Advantage 8 Leverages Moore’s Law for I/O Bandwidth Balances memory and SSD for peak performance Uses rotating disks for capacity Shares data across I/O Channels Reduces dependence on backend bandwidth for peak performance Makes the data available to multiple clients without data replication Reliability No single point of failure Data includes error correction for inexpensive cache and backend storage Usability Clients can use the I/O channel without modification

The Benefits 9 Deliver a solution that provides dramatic computation and application programming improvements Computational improvements through superior I/O Application programming improvements Compute server usage Scalability Reliability Green I/O wait time reduction Huge Shared Cache Reliable Memory Paradigms Scalable ISV environments

Solution Summary 10 Extreme Performance Dramatic performance improvements in High Performance Computing modeling and simulation applications Performance scales linearly with increase in data volumes Standards-based NFS frontend, works equally well with Linux, Unix and Windows Works transparently with existing applications with no change in application management Cost Effective Backend file system is scaled to average performance Cache manages the peak load Value Takes advantage of the massive availability and scalability of non- proprietary hardware Delivers higher performance than similarly configured systems providing budget relief, increased performance per dollar, as well as energy consumption and footprint