FutureGrid Connection to Comet Testbed and On Ramp as a Service Geoffrey Fox Indiana University Infra structure.

Slides:

Advertisements

Similar presentations

Cloud computing is used to describe a variety of computing concepts that involve a large number of computers connected through a real-time communication.

Advertisements

FutureGrid related presentations at TG and OGF Sun. 17th: Introduction to FutireGrid (OGF) Mon. 18th: Introducing to FutureGrid (TG) Tue. 19th –Educational.

FutureGrid UAB Meeting XSEDE13 San Diego July

Cloudmesh Resource Shifting 1 2. Cloudmesh: from IaaS(NaaS) to Workflow (Orchestration) Workflow Virtual Cluster Components Infrastructure iPython (Pegasus)

Clouds from FutureGrid’s Perspective April Geoffrey Fox Director, Digital Science Center, Pervasive.

COMS E Cloud Computing and Data Center Networking

FutureGrid Image Repository: A Generic Catalog and Storage System for Heterogeneous Virtual Machine Images Javier Diaz, Gregor von Laszewski, Fugang Wang,

Cloudmesh: Software Defined Distributed Systems as a Service SDDSaaS January BigDat 2015: International Winter School on Big Data Tarragona, Spain,

Big Data Open Source Software and Projects ABDS in Summary VI: Layer 6 Part 2 Data Science Curriculum March Geoffrey Fox

INTRODUCTION TO CLOUD COMPUTING CS 595 LECTURE 4.

Cloudmesh a Gentle Overview Gregor von Laszewski Sep. 2014

Cloudmesh: Software Defined Distributed Systems as a Service SDDSaaS Workshop on the Development of a Next-Generation, Interoperable, Federated Network.

Design Discussion Rain: Dynamically Provisioning Clouds within FutureGrid Geoffrey Fox, Andrew J. Younge, Gregor von Laszewski, Archit Kulshrestha, Fugang.

Introduction to Cloud Computing

Accessing and Managing Multiple Clouds (Infrastructures) with Cloudmesh June BigSystem Software-Defined Ecosystems at HPDC Vancouver Canada.

A Brief Overview by Aditya Dutt March 18 th ’ Aditya Inc.

Big Data and Clouds: Challenges and Opportunities NIST January Geoffrey Fox

Eucalyptus on FutureGrid: A case for Eucalyptus 3 Sharif Islam, Javier Diaz, Geoffrey Fox Gregor von Laszewski Indiana University.

LARGE SCALE DEPLOYMENT OF DAP AND DTS Rob Kooper Jay Alemeda Volodymyr Kindratenko.

Cloud Computing. What is Cloud Computing? Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable.

Accessing and Managing Multiple Clouds (Infrastructures) with Cloudmesh June BigSystem Software-Defined Ecosystems at HPDC Vancouver Canada.

Science Clouds and FutureGrid’s Perspective June Science Clouds Workshop HPDC 2012 Delft Geoffrey Fox

Selected lessons learned from FutureGrid resulting in a toolkit for ComputingTestbedaaS: Cloudmesh HPDS 2014, Halifax, CA Gregor von Laszewski Geoffrey.

Data Science at Digital Science October Geoffrey Fox Judy Qiu

Large Scale Sky Computing Applications with Nimbus Pierre Riteau Université de Rennes 1, IRISA INRIA Rennes – Bretagne Atlantique Rennes, France

Niagara Framework in the Clouds Scott Boehm. … what the heck does that mean??

Image Generation and Management on FutureGrid CTS Conference 2011 Philadelphia May Geoffrey Fox

RAIN: A system to Dynamically Generate & Provision Images on Bare Metal by Application Users Presented by Gregor von Laszewski Authors: Javier Diaz, Gregor.

FutureGrid Computing Testbed as a Service Overview July Geoffrey Fox for FutureGrid Team

SALSASALSASALSASALSA FutureGrid Venus-C June Geoffrey Fox

FutureGrid Computing Testbed as a Service NSF Presentation NSF April Geoffrey Fox for FutureGrid Team

FutureGrid Computing Testbed as a Service for Condo_of_Condos Internet 2 panel April Jose Fortes for FutureGrid Team.

Big Data Open Source Software and Projects ABDS in Summary IV: Level 7 I590 Data Science Curriculum August Geoffrey Fox

Internet of Things (Smart Grid) Storm Archival Storage – NOSQL like Hbase Streaming Processing (Iterative MapReduce) Batch Processing (Iterative MapReduce)

Design Discussion Rain: Dynamically Provisioning Clouds within FutureGrid PI: Geoffrey Fox*, CoPIs: Kate Keahey +, Warren Smith -, Jose Fortes #, Andrew.

Computing Research Testbeds as a Service: Supporting large scale Experiments and Testing SC12 Birds of a Feather November.

Recipes for Success with Big Data using FutureGrid Cloudmesh SDSC Exhibit Booth New Orleans Convention Center November Geoffrey Fox, Gregor von.

Future Grid Future Grid Overview. Future Grid Future GridFutureGridFutureGrid The goal of FutureGrid is to support the research that will invent the future.

SALSASALSASALSASALSA Digital Science Center February 12, 2010, Bloomington Geoffrey Fox Judy Qiu

Vignesh Ravindran Sankarbala Manoharan. Infrastructure As A Service (IAAS) is a model that is used to deliver a platform virtualization environment with.

Document Name CONFIDENTIAL Version Control Version No.DateType of ChangesOwner/ Author Date of Review/Expiry The information contained in this document.

Cloud Computing Shannon McManus Michael Weihert. What is Cloud Computing?

FutureGrid Computing Testbed as a Service EGI Technical Forum 2013 Madrid Spain September Geoffrey Fox and Gregor.

Directions in eScience Interoperability and Science Clouds June Interoperability in Action – Standards Implementation.

Grappling Cloud Infrastructure Services with a Generic Image Repository Javier Diaz Andrew J. Younge, Gregor von Laszewski, Fugang.

Big Data Open Source Software and Projects ABDS in Summary II: Layer 5 I590 Data Science Curriculum August Geoffrey Fox

Indiana University Faculty Geoffrey Fox, David Crandall, Judy Qiu, Gregor von Laszewski Data Science at Digital Science Center.

Cloud Computing Service Architectures V. Arun College of Computer Science University of Massachusetts Amherst 1.

In Depth Azure StackIn Depth Azure Stack Resource Providers Damian Flynn MVP Daniel Savage Microsoft.

CLOUD ARCHITECTURE Many organizations and researchers have defined the architecture for cloud computing. Basically the whole system can be divided into.

Organizations Are Embracing New Opportunities

Private Public FG Network NID: Network Impairment Device

Digital Science Center II

Principles of Computer Security

Bridges and Clouds Sergiu Sanielevici, PSC Director of User Support for Scientific Applications October 12, 2017 © 2017 Pittsburgh Supercomputing Center.

Some Remarks for Cloud Forward Internet2 Workshop

FutureGrid Computing Testbed as a Service

Digital Science Center Overview

Cloud Computing Dr. Sharad Saxena.

Dr. John P. Abraham Professor, Computer Engineering UTPA

I590 Data Science Curriculum August

Versatile HPC: Comet Virtual Clusters for the Long Tail of Science SC17 Denver Colorado Comet Virtualization Team: Trevor Cooper, Dmitry Mishin, Christopher.

Data Science Curriculum March

Cloud DIKW based on HPC-ABDS to integrate streaming and batch Big Data

Clouds from FutureGrid’s Perspective

Gregor von Laszewski Indiana University

Cloud Computing: Concepts

Using and Building Infrastructure Clouds for Science

PolarGrid and FutureGrid

Virtual Clusters Production Experience

Presentation transcript:

FutureGrid Connection to Comet Testbed and On Ramp as a Service Geoffrey Fox Indiana University Infra structure IaaS  Software Defined Computing (virtual Clusters)  Hypervisor, Bare Metal  Operating System Platform PaaS  Cloud e.g. MapReduce  HPC e.g. PETSc, SAGA  Computer Science e.g. Compiler tools, Sensor nets, Monitors Network NaaS  Software Defined Networks  OpenFlow GENI Software (Application Or Usage) SaaS  CS Research Use e.g. test new compiler or storage model  Class Usages e.g. run GPU & multicore  Applications

Deployment Automation Support deployment of a given program on multiple platforms – interoperability Test on small interactive isolatable platforms and deploy on comet Support virtual clusters – a job that involves multiple nodes possibly on different distributed platforms -- federation Support clouds and bare metal Support adding (cloud bursting) and moving nodes Use templated machine images – “Software defined computing or systems” – jobs defined abstractly Obvious security issues Keep up with continued dramatic change

CloudMesh Architecture

Evolution of Bare metal/Hypervisor provisioning Many changing approaches and many gotchas – E.g. Approaches only work on some linux’s and some network adapter software releases FutureGrid (early) – Uses XCAT as part of FG Rain. It allows us to provision images on virtual machines, but also bare metal. We use templated images that can easily be transformed to run on different IaaS as well as bare metal FutureGrid Cloudmesh bare metal abstraction – Abstract the bare metal provisioners to a very simple set of functionality so they are easier to integrate in higher level tools – Provide different backends – Currently uses Cloudmesh teefaa – Planned: OpenStack bare metal, cobbler, XCAT, …

Technologies classified by TrippleO OpenStack on openstack Orchestration ~ virtual cluster slides/TripleO%20Overview.pdf slides/TripleO%20Overview.pdf

Some Integrated Resources Core FutureGrid machines including Sierra at SDSC with Bare Metal, OpenStack and Eucalyptus Azure and Amazon OpenStack based HPCloud Working on OpenCirrus cloud (Tashi) FutureGrid metrics project links metrics to XSEDE Cloudmesh Python based and supporting Libcloud, OpenStack, Azure and AWS API’s Extend to comet and others over next year