We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byPhoenix Gooch
Modified about 1 year ago
TEMPLATE DESIGN © High Performance Molecular Dynamics in Cloud Infrastructure with SR-IOV and GPUDirect Andrew J. Younge *, John Paul Walters +, Geoffrey C. Fox* Introduction BenchmarksResults Conclusion References At present we stand at the inevitable intersection between High Performance Computing (HPC) and Clouds. Various platform tools such as Hadoop and MapReduce, among others have already percolated into data intensive computing within HPC . Alternatively, there are efforts to support traditional HPC-centric scientific computing applications in virtualized Cloud infrastructure. The reasons for supporting parallel computation on Cloud infrastructure is bounded only by the advantages of Cloud computing itself . For users, this includes features such as dynamic scalability, specialized operating environments, simple management interfaces, fault tollarance, and enhanced quality of service, to name a few. The growing importance of supporting advanced scientific computing using cloud infrastructure can be seen by a variety of new efforts, including the NSF-funded XSEDE Comet resource at SDSC . Reluctantly, there exists a past notion that virtualization used in today’s Cloud infrastructure is inherently inefficient. Historically, Cloud infrastructure has also done little to provide the necessary advanced hardware capabilities that have become almost mandatory in Supercomputers today, most notably advanced GPUs and high-speed, low-latency interconnects. The result of these notions has hindered the use of virtualized environments for parallel computation, where performance must be paramount. Recent advances in hypervisor performance  coupled with the newfound availably of HPC hardware in virtual machines analogous to the most powerful supercomputers used today, we see can see the formation of a High Performance Cloud infrastructure. While our previous advanced in this are have focused on single-node advancements, it is now imparative to ensure real-world applications can also operate at scale. Furthermore, the tight and exact integration into an open source Cloud infrastructure framework such as OpenStack also becomes a critical next step.  S. Jha, J. Qiu, A. Luckow, P. K. Mantha, and G. C. Fox, “A tale of two data-intensive paradigms: Applications, abstractions, and architectures,” in Proceedings of the 3rd International Congress on Big Data (BigData 2014),  M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, “A view of cloud computing,” Commun. ACM, vol. 53, no. 4, pp. 50–58, Apr  M. Norman and R. Moore, “NSF awards 12 million to sdsc to deploy comet supercomputer,” Web page,  A. J. Younge, R. Henschel, J. T. Brown, G. von Laszewski, J. Qiu, and G. C. Fox, “Analysis of Virtualization Technologies for High Performance Computing Environments,” in Proceedings of the 4th International Conference on Cloud Computing (CLOUD 2011). Washington, DC: IEEE, July  J. P. Walters, A. J. Younge, D.-I. Kang, K.-T. Yao, M. Kang, S. P. Crago, and G. C. Fox, “GPU-Passthrough Performance: A Comparison of KVM, Xen, VMWare ESXi, and LXC for CUDA and OpenCL Applications,” in Proceedings of the 7th IEEE International Conference on Cloud Computing (CLOUD 2014), IEEE. Anchorage, AK: IEEE,  J. Jose, M. Li, X. Lu, K. C. Kandalla, M. D. Arnold, and D. K. Panda, “SR-IOV support for virtualization on infiniband clusters: Early experience,” in Cluster Computing and the Grid, IEEE International Symposium on, 2013, pp. 385–392.  M. Musleh, V. Pai, J. P. Walters, A. J. Younge, and S. P. Crago, “Bridging the Virtualization Performance Gap for HPC using SR-IOV for InfiniBand,” in Proceedings of the 7th IEEE International Conference on Cloud Computing (CLOUD 2014), IEEE. Anchorage, AK: IEEE,  W. M. Brown, “GPU acceleration in LAMMPS  J. Anderson, A. Keys, C. Phillips, T. Dac Nguyen, and S. Glotzer, “HOOMD-Blue, general-purpose many-body dynamics on the GPU,” in APS Meeting Abstracts, vol. 1, 2010, p * School of Informatics & Computing, Indiana University 901 E. 10th St., Bloomington, IN U.S.A. + Information Sciences Institute, University of Southern California 3811 North Fairfax Drive, Suite 200, Arlington, VA U.S.A. LAMMPS HOOMD-Blue Focus Areas Hypervisor Performance Virtualization can operate with near-native performance. IO Virtualization Leverage VT-d/IOMMU extensions to pass PCI-based hardware directly to a guest VM. GPUs and Accelerators Utilize PCI Passthrough in to provide GPUs to VMs Many hypervisors now able to support GPU Passthrough – we use KVM for best performance. High Speed Interconnects Using SR-IOV, we can create multiple VFs from a single PCI devicee, each assigned directly to a VM. Use Mellanox ConnectX3 VPI InfiniBand. QDR/FDR InfiniBand now possible within Cloud IaaS! OpenStack Integration Integrate virtualization advances to the OpenStack Cloud IaaS. Prototype available, some features available today Historically running advanced scientific applications in a virtualized infrastructure has been limited by both performance and advanced hardware availability Recent advancements allow for the use of both GPUs and InfiniBand fabric to be leveraged directly in VMs LAMMPS and HOOMD represent Molecular Dynamics tools commonly used on the most powerful supercomputers Virtualized performance for both applications at near-native 96.7% and 99.3% efficiency for LAMMPS LJ 2048k and RHODO 512k simulations 98.5% efficiency for HOOMD LJ 256k simulation Support for new GPUDirect RDMA features in virtualized system Large-scale virtualized Cloud Infrastructure can now support many of the same advanced scientific computations that are commonly found running on today’s supercomputers.
2010 FutureGrid User Advisory Meeting Architecture Roadmap Long term vision 10:00-10:45, Monday, August 2, 2010 Pittsburgh, PA Gregor von Laszewski Representing.
FutureGrid UAB Meeting XSEDE13 San Diego July
Cloud Computing Cloud Computing Overview of Distributed Computing.
FutureGrid related presentations at TG and OGF Sun. 17th: Introduction to FutireGrid (OGF) Mon. 18th: Introducing to FutureGrid (TG) Tue. 19th –Educational.
Overview of the FutureGrid Software Presented by Gregor von Laszewski Indiana University Workshop on High Performance Applications of Cloud and Grid Tools.
FutureGrid and US Cyberinfrastructure Collaboration with EU Symposium on transatlantic EU-U.S. cooperation in the field of large scale research infrastructures.
Https://portal.futuregrid.org What FutureGrid Can Do for You? TeraGrid’11 BOF Session 1 Salt Lake City, Utah July 20 th 2011.
1 Towards an Open Service Framework for Cloud-based Knowledge Discovery Domenico Talia ICAR-CNR & UNIVERSITY OF CALABRIA, Italy Cloud.
INDIANAUNIVERSITYINDIANAUNIVERSITY GENI Global Environment for Network Innovation James Williams Director – International Networking Director – Operational.
The VM deployment process has 3 major steps: 1.The client queries the VM repository, sending a list of criteria describing a workspace. The repository.
Distributed Computing Dr. Eng. Ahmed Moustafa Elmahalawy Computer Science and Engineering Department.
Operating Systems Part I: Introduction. “I think that there is a world market for five computers” - Thomas J. Watson (1945)
Partition and Isolate: Approaches for Consolidating HPC and Commodity Workloads Jack Lange Assistant Professor University of Pittsburgh.
Sponsors and Acknowledgments This work is supported in part by the National Science Foundation under Grants No. OCI , IIP and CNS
RMS and Scheduling for Future Generation Grids Ramin Yahyapour University Dortmund Leader CoreGRID Institute on Resource Management and Scheduling CoreGRID.
Virtualization Technique Virtualization Technique System Virtualization I/O Virtualization.
The Future of Computing Performance: Samuel H. Fuller, Chair March 22, 2011 Computer Science and Telecommunications Board (CSTB) National Research Council.
Assembling the EVOp Infrastructure Yehia El-khatib, Gordon S. Blair School of Computing & Communications Lancaster University.
The first-generation Cell Broadband Engine (BE) processor is a multi-core chip comprised of a 64-bit Power Architecture processor core and eight synergistic.
Tools and Services for Data Intensive Research Roger Barga, Architect eXtreme Computing Group, Microsoft Research An Elephant Through the Eye of a Needle.
From Sandbox to Playground: Virtual Environments and Quality of Service in the Grids Kate Keahey Argonne National Laboratory.
Grid Computing Evolution and Challenges for Resilience, Performance and Scalability Luca Simoncini University of Pisa, Italy July 2, 2005WS on Grid Computing.
KIT – The cooperation of Forschungszentrum Karlsruhe GmbH und Universität Karlsruhe (TH) The OpenCirrus TM Project: A global Testbed for Cloud.
C YBERINFRASTRUCTURE C ENTER FOR P OLAR S CIENCE (CICPS) Cyberinfrastructure for Remote Sensing of Ice Sheets Dr. Linda Hayden Dr. Geoffrey Fox Dr. Prasad.
DC-API: Unified API for Desktop Grid Systems Gábor Gombás MTA SZTAKI.
INFSO-SSA International Collaboration to Extend and Advance Grid Education OGF User Forum Boston, USA, 23 th – xx th February 2008 tInfrastructure.
Agent Based Software Development Michael Luck, Ronald Ashri and Mark dInverno Chapter 4: Methodologies and Modeling Languages.
Converged Datatech introduces Converged infrastructure.
© 2016 SlidePlayer.com Inc. All rights reserved.