1. FutureGrid: an experimental, high-performance grid testbed Craig Stewart Executive Director, Pervasive Technology Institute Indiana University stewart@iu.edu www.futuregrid.org 3 March 2010

2. License terms Please cite as: Stewart, C.A. 2010. FutureGrid: an experimental, high- performance grid testbed (TGQ). (Presentation) TeraGrid Quarterly Meeting (Tampa Bay, FL, 1-3 Mar 2010). Available from: http://hdl.handle.net/2022/13911 http://hdl.handle.net/2022/13911 Except where otherwise noted, by inclusion of a source url or some other note, the contents of this presentation are © by the Trustees of Indiana University. This content is released under the Creative Commons Attribution 3.0 Unported license (http://creativecommons.org/licenses/by/3.0/). This license includes the following terms: You are free to share – to copy, distribute and transmit the work and to remix – to adapt the work under the following conditions: attribution – you must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). For any reuse or distribution, you must make clear to others the license terms of this work. 2

3. Geographic View

4. FutureGrid will… put the “science” back in the computer science of grid computing by enabling replicable experiments Be a grid testbed using virtualization technologies to be whatever you need, when you need a robustly managed simulation environment or testbed to support the development and early use in science of new technologies at all levels of the software stack: from networking to middleware to scientific applications.

5. FutureGrid Partners Indiana University Purdue University San Diego Supercomputer Center at University of California San Diego University of Chicago/Argonne National Labs University of Florida University of Southern California Information Sciences Institute University of Tennessee Knoxville University of Texas at Austin/Texas Advanced Computing Center University of Virginia ZIH and GWT-TUD, Technische Universtität Dresden.

6. Other Important Collaborators Other Important Collaborators Early users from an application and computer science perspective and from both research and education Grid5000/Aladin and D-Grid in Europe Commercial partners such as – Eucalyptus – Microsoft (Dryad + Azure) – Note Azure external to FutureGrid

7. Key Management PI. Geoffrey Fox is the PI, and have overall responsibility for the project as a whole. Fox will be the final arbiter of any decisions that cannot be reached by a consensus approach. Co-PIs. Kate Keahey, Warren Smith, Jose Fortes, and Andrew Grimshaw Executive Investigator. Craig Stewart will serve as executive director, responsible for operational management of FutureGrid. Chief Architect. Gregor von Laszewski (who joined IU on 22 July 2009) will serve as the chief architect for FutureGrid. RP Lead. Joe Rinkovsky will be the RP lead Project Manager. Gary Miksik will serve 0.5 FTE as project manager for FutureGrid, and have management of the WBS, preparation of reports, and collection of responses to requests for information from the NSF as his primary job responsibilities.

8. FutureGrid Usage Scenarios Developers of end-user applications who want to develop new applications in cloud or grid environments, including analogs of commercial cloud environments such as Amazon or Google. – Is a Science Cloud for me? Developers of end-user applications who want to experiment with multiple hardware environments. Grid middleware developers who want to evaluate new versions of middleware or new systems. Networking researchers who want to test and compare different networking solutions in support of grid and cloud applications and middleware. (Some types of networking research will likely best be done via through the GENI program.) Interest in performance requires ability to deploy outside VM environments

9. Compute Hardware System type# CPUs# CoresTFLOPSTotal RAM (GB) Secondary Storage (TB) Site Status Dynamically configurable systems IBM iDataPlex2561024113072339*IU New System Dell PowerEdge19211528 15TACC New System IBM iDataPlex16867272016120UC New System IBM iDataPlex1686727268872SDSC Existing System Subtotal7843520338928546 Systems not dynamically configurable Cray XT5m16867261344339*IU New System Shared memory system TBD 404804640339*IU New System 4Q2010 Cell BE Cluster480164IU Existing System IBM iDataPlex6425627681UF New System High Throughput Cluster 1923844192PU Existing System Subtotal46818721730081 Total125253925011936547

10. Storage Hardware System TypeCapacity (TB)File SystemSiteStatus DDN 9550 (Data Capacitor) 339LustreIUExisting System DDN 6620120GPFSUCNew System SunFire x417072Lustre/PVFSSDSCNew System Dell MD300030NFSTACCNew System

11. Logical Diagram

12. Network Impairments Device Spirent XGEM Network Impairments Simulator for jitter, errors, delay, etc Full Bidirectional 10G w/64 byte packets up to 15 seconds introduced delay (in 16ns increments) 0-100% introduced packet loss in.0001% increments Packet manipulation in first 2000 bytes up to 16k frame size TCL for scripting, HTML for human configuration More easily replicable than keeping teenagers around the house……

13. Selected FutureGrid Timeline October 1 2009 Project Started October 2-3 2009 First All Hands Meeting November 16-19 SC09 Demo/F2F Committee Meetings January 2010 First Science Board Meeting March 2010 FutureGrid network complete April – several major new systems have passed acceptance, doing some early work September 2010 All hardware (except anticipated shared memory system) accepted October 1 2011 FutureGrid allocatable via TeraGrid process – first two years by user/science board led by Andrew Grimshaw

14. System Milestones New IBM Systems – Delivery: December 2009 – Acceptance: March 2010 IU SYSTEM ACCEPTED – Available for Use: April 2010 Cray – Delivery: December 2009 – Acceptance: March 2010 ACCEPTED Dell System – Delivery: January 2010 – Acceptance: March 2010 – Available for Use: April 2010 Existing IU iDataPlex – Move to SDSC: January 2010 DONE – Available for Use: March 2010 Storage Systems (Sun & DDN) – Delivery: October 2009 – Available for Use: December 2009

15. Future Grid Users Application/Scientific users System administrators Software developers Testbed users Performance modelers Educators Students Supported by FutureGrid Infrastructure & Software offerings http://futuregrid.org 15

16. Objectives: Software significant extensions to existing software existing open-source software open-source, integrated suite of software to – instantiate and execute grid and cloud experiments. – perform an experiment – collect the results – tools for instantiating a test environment, Torque, MOAB, xCAT, bcfg, and Pegasus, Inca, ViNE, a number of other tools from our partners and the open source community Portal to interact – Benchmarking http://futuregrid.org 16

17. FG Stratosphere Objective – Higher than a particular cloud – Provides all mechanisms to provision a cloud on a given FG hardware – Allows the management of reproducible experiments – Allows monitoring of the environment and the results Risks – Lots of software – Possible multiple path to do the same thing Good news – We know about different solutions and have identified a very good plan with risk mitigation plans http://futuregrid.org 17

18. Rain Runtime Adaptable InsertioN Service Objective – Provide dynamic provisioning – Running outside virtualization – Cloud neutral Nimbus, Eucalyptus, … – Future oriented Dryad … Risks – Some frameworks (e.g. MS) are more complex to provision http://futuregrid.org 18

19. Dynamic Provisioning Change underlying system to support current user demands Linux, Windows, Xen, Nimbus, Eucalyptus Stateless images Shorter boot times Easier to maintain Stateful installs Windows Use moab to trigger changes and xCAT to manage installs 19 http://futuregrid.org

20. Command line fg-deploy-image – host name – image name – start time – end time – label name fg-add – label name – framework hadoop – version 1.0 Deploys an image on a host Adds a feature to a deployed image http://futuregrid.org 20

21. xCAT and Moab xCAT uses installation infrastructure to perform installs creates stateless Linux images changes the boot configuration of the nodes remote power control and console (IPMI) Moab meta-schedules over resource managers  TORQUE and Windows HPC control nodes through xCAT  changing the OS  remote power control 21 http://futuregrid.org

22. Experiment Manager Objective – Manage the provisioning for reproducible experiments – Coordinate workflow of experiments – Share workflow and experiment images – Minimize space through reuse Risk – Images are large – Users have different requirements and need different images http://futuregrid.org 22

23. Integration within TeraGrid / TeraGrid XD Sure it’s part of TeraGrid Allocation: separate from TG processes for two years It is a very exciting project, but IIWIIAIAWIA We are looking for early adopters!

25. Thanks! Questions?