FutureGrid: an experimental, high-performance grid testbed Craig Stewart Executive Director, Pervasive Technology Institute Indiana University

Slides:



Advertisements
Similar presentations
Virtualization, Cloud Computing, and TeraGrid Kate Keahey (University of Chicago, ANL) Marlon Pierce (Indiana University)
Advertisements

FutureGrid and US Cyberinfrastructure Collaboration with EU Symposium on transatlantic EU-U.S. cooperation in the field of large scale research infrastructures.
FutureGrid Overview NSF PI Science of Cloud Workshop Washington DC March Geoffrey Fox
Cosmic Issues and Analysis of External Comments on FutureGrid TG11 Salt Lake City July Geoffrey Fox
Clouds from FutureGrid’s Perspective April Geoffrey Fox Director, Digital Science Center, Pervasive.
Future Grid Introduction March MAGIC Meeting Gregor von Laszewski Community Grids Laboratory, Digital Science.
1 Supplemental line if need be (example: Supported by the National Science Foundation) Delete if not needed. Supporting Polar Research with National Cyberinfrastructure.
Student Visits August Geoffrey Fox
Jefferson Ridgeway 2, Ifeanyi Rowland Onyenweaku 3, Gregor von Laszewski 1*, Fugang Wang 1 1* Indiana University, Bloomington, IN 47408, U.S.A.,
Overview Presented at OGF31 Salt Lake City, July 2011 Geoffrey Fox, Gregor von Laszewski, Renato Figueiredo Contact:
SALSASALSASALSASALSA Digital Science Center June 25, 2010, IIT Geoffrey Fox Judy Qiu School.
NSF Vision and Strategy for Advanced Computational Infrastructure Vision: NSF Leadership in creating and deploying a comprehensive portfolio…to facilitate.
FutureGrid Summary TG’10 Pittsburgh BOF on New Compute Systems in the TeraGrid Pipeline August Geoffrey Fox
FutureGrid Overview Bloomington Indiana January FutureGrid Collaboration Presented by Geoffrey Fox
SC2010 Gregor von Laszewski (*) (*) Assistant Director of Cloud Computing, CGL, Pervasive Technology Institute.
FutureGrid Summary FutureGrid User Advisory Board TG’10 Pittsburgh August Geoffrey Fox
Eucalyptus on FutureGrid: A case for Eucalyptus 3 Sharif Islam, Javier Diaz, Geoffrey Fox Gregor von Laszewski Indiana University.
FutureGrid: A Distributed High Performance Test-bed for Clouds Andrew J. Younge Indiana University
FutureGrid Overview David Hancock HPC Manger Indiana University.
FutureGrid: an experimental, high-performance grid testbed Craig Stewart Executive Director, Pervasive Technology Institute Indiana University
Clouds and FutureGrid MSI-CIEC All Hands Meeting SDSC January Geoffrey Fox
FutureGrid Overview CTS Conference 2011 Philadelphia May Geoffrey Fox
Overview of Cyberinfrastructure Northeastern Illinois University Cyberinfrastructure Day August Geoffrey Fox
FutureGrid SOIC Lightning Talk February Geoffrey Fox
Distributed FutureGrid Clouds for Scalable Collaborative Sensor-Centric Grid Applications For AMSA TO 4 Sensor Grid Technical Interchange Meeting By Anabas,
Science of Cloud Computing Panel Cloud2011 Washington DC July Geoffrey Fox
FutureGrid and US Cyberinfrastructure Collaboration with EU Symposium on transatlantic EU-U.S. cooperation in the field of large scale research infrastructures.
Experimenting with FutureGrid CloudCom 2010 Conference Indianapolis December Geoffrey Fox
PolarGrid Geoffrey Fox (PI) Indiana University Associate Dean for Graduate Studies and Research, School of Informatics and Computing, Indiana University.
Advanced Computing and Information Systems laboratory The case for UF in PRAGMA Jose Fortes (also on behalf of Renato Figueiredo and Reed Beaman)
Science Clouds and FutureGrid’s Perspective June Science Clouds Workshop HPDC 2012 Delft Geoffrey Fox
Gregor von Laszewski*, Geoffrey C. Fox, Fugang Wang, Andrew Younge, Archit Kulshrestha, Greg Pike (IU), Warren Smith, (TACC) Jens Vöckler (ISI), Renato.
FutureGrid Overview Geoffrey Fox
FutureGrid: an experimental, high-performance grid testbed Craig Stewart Executive Director, Pervasive Technology Institute Indiana University
FutureGrid Design and Implementation of a National Grid Test-Bed David Hancock – HPC Manager - Indiana University Hardware & Network.
Future Grid FutureGrid Overview Dr. Speaker. Future Grid Future GridFutureGridFutureGrid The goal of FutureGrid is to support the research on the future.
FutureGrid Overview Geoffrey Fox
FutureGrid: an experimental, high-performance grid testbed Craig Stewart Executive Director, Pervasive Technology Institute Indiana University
FutureGrid Dynamic Provisioning Experiments including Hadoop Fugang Wang, Archit Kulshrestha, Gregory G. Pike, Gregor von Laszewski, Geoffrey C. Fox.
What’s Hot in Clouds? Analyze (superficially) the ~140 Papers/Short papers/Workshops/Posters/Demos in CloudCom Each paper may fall in more than one category.
Future Grid FutureGrid Overview Geoffrey Fox SC09 November
FutureGrid SC10 New Orleans LA IU Booth November Geoffrey Fox
FutureGrid Overview Geoffrey Fox
Future Grid Future Grid All Hands Meeting Introduction Indianapolis October Geoffrey Fox
FutureGrid SOIC Lightning Talk February Geoffrey Fox
FutureGrid Cyberinfrastructure for Computational Research.
Building Effective CyberGIS: FutureGrid Marlon Pierce, Geoffrey Fox Indiana University.
RAIN: A system to Dynamically Generate & Provision Images on Bare Metal by Application Users Presented by Gregor von Laszewski Authors: Javier Diaz, Gregor.
SALSASALSASALSASALSA FutureGrid Venus-C June Geoffrey Fox
Research in Grids and Clouds and FutureGrid Melbourne University September Geoffrey Fox
FutureGrid TeraGrid Science Advisory Board San Diego CA July Geoffrey Fox
FutureGrid Overview Geoffrey Fox
Bio Gregor von Laszewski is conducting state-of-the-art work in Cloud computing and GreenIT at Indiana University as part of the Future Grid project. During.
Tutorial Presented at TG2011 Geoffrey Fox, Gregor von Laszewski, Renato Figueiredo, Kate Keahey, Andrew Younge Contact:
FutureGrid BOF Overview TG 11 Salt Lake City July Geoffrey Fox
FutureGrid NSF September Geoffrey Fox
Design Discussion Rain: Dynamically Provisioning Clouds within FutureGrid PI: Geoffrey Fox*, CoPIs: Kate Keahey +, Warren Smith -, Jose Fortes #, Andrew.
1 NSF/TeraGrid Science Advisory Board Meeting July 19-20, San Diego, CA Brief TeraGrid Overview and Expectations of Science Advisory Board John Towns TeraGrid.
Computing Research Testbeds as a Service: Supporting large scale Experiments and Testing SC12 Birds of a Feather November.
Future Grid Future Grid Overview. Future Grid Future GridFutureGridFutureGrid The goal of FutureGrid is to support the research that will invent the future.
SALSASALSASALSASALSA Digital Science Center February 12, 2010, Bloomington Geoffrey Fox Judy Qiu
IEEE Cloud 2011 Panel: Opportunities of Services Business in Cloud Age Fundamental Research Challenges for Cloud-based Service Delivery Gregor von Laszewski.
Grappling Cloud Infrastructure Services with a Generic Image Repository Javier Diaz Andrew J. Younge, Gregor von Laszewski, Fugang.
Building on virtualization capabilities for ExTENCI Carol Song and Preston Smith Rosen Center for Advanced Computing Purdue University ExTENCI Kickoff.
Lizhe Wang, Gregor von Laszewski, Jai Dayal, Thomas R. Furlani
Clouds , Grids and Clusters
FutureGrid: a Grid Testbed
Clouds from FutureGrid’s Perspective
FutureGrid and Applications
PolarGrid and FutureGrid
Presentation transcript:

FutureGrid: an experimental, high-performance grid testbed Craig Stewart Executive Director, Pervasive Technology Institute Indiana University

FutureGrid will… put the “science” back in the computer science of grid computing. be an “experiment factory.” be like a schmoo, only instrumented and better – a grid testbed using virtualization technologies to be whatever you need, when you need. enable advances in science and engineering through collaborative evolution of science applications and related software. be a robustly managed simulation environment or testbed to support the development and early use in science of new technologies at all levels of the software stack: from networking to middleware to scientific applications. not be a production cloud environment. 2

Geographic View

FutureGrid Usage Scenarios Developers of end-user applications who want to develop new applications in cloud or grid environments, including analogs of commercial cloud environments such as Amazon or Google. –Is a Science Cloud for me? Developers of end-user applications who want to experiment with multiple hardware environments. Grid middleware developers who want to evaluate new versions of middleware or new systems. Networking researchers who want to test and compare different networking solutions in support of grid and cloud applications and middleware. (Some types of networking research will likely best be done via through the GENI program.) Interest in performance requires ability to deploy outside VM environments.

FutureGrid Partners Indiana University Purdue University San Diego Supercomputer Center at University of California San Diego University of Chicago/Argonne National Labs University of Florida University of Southern California Information Sciences Institute University of Tennessee Knoxville University of Texas at Austin/Texas Advanced Computing Center University of Virginia Center for Information Services and GWT-TUD from Technische Universtität Dresden. Other important collaborators: –Early users from an application and computer science perspective and from both research and education –Grid5000/ALADDIN and D-Grid in Europe –Commercial partners such as Eucalyptus Microsoft (Dryad + Azure) – Note Azure external to FutureGrid like GPU systems

Compute Hardware System type# CPUs# CoresTFLOPS Total RAM (GB) Secondary Storage (TB) Site Status Dynamically configurable systems IBM iDataPlex *IU New System Dell PowerEdge TACC New System IBM iDataPlex UC New System IBM iDataPlex SDSC Existing System Subtotal Systems not dynamically configurable Cray XT5m *IU New System Shared memory system TBD *IU New System 4Q2010 Cell BE Cluster480164IU Existing System IBM iDataPlex UF New System High Throughput Cluster PU Existing System Subtotal Total

Logical Diagram

Network Impairments Device Spirent XGEM Network Impairments Simulator for jitter, errors, delay, etc Full Bidirectional 10G w/64 byte packets up to 15 seconds introduced delay (in 16ns increments) 0-100% introduced packet loss in.0001% increments Packet manipulation in first 2000 bytes up to 16k frame size TCL for scripting, HTML for human configuration More replicable than keeping teenagers around the house……

Selected FutureGrid Timeline October Project Started October First All Hands Meeting November SC09 Demo/F2F Committee Meetings January 2010 First Science Board Meeting March 2010 FutureGrid network complete March 2010 FutureGrid Annual Meeting September 2010 All hardware (except anticipated shared memory system) accepted October FutureGrid allocatable via TeraGrid process – first two years by user/science board led by Andrew Grimshaw

Key Management PI. Geoffrey Fox will be the PI, and have overall responsibility for the project as a whole. Fox will be the final arbiter of any decisions that cannot be reached by a consensus approach. Co-PIs. Kate Keahey, Warren Smith, Jose Fortes, and Andrew Grimshaw. Executive Investigator. Craig Stewart will serve as executive director, responsible for operational management of FutureGrid. Chief Architect. Gregor von Laszewski (who joined IU on 22 July 2009) will serve as the chief architect for FutureGrid. RP Lead. Responsible for interfacing between FutureGrid and the TeraGrid (and later TeraGrid XD). Project Manager. Gary Miksik will serve 0.5 FTE as project manager for FutureGrid, and have management of the WBS, preparation of reports, and collection of responses to requests for information from the NSF as his primary job responsibilities.

And one slide about PTI