Hosting Cloud, HPC and Grid Educational Activities on FutureGrid Renato Figueiredo – U. of Florida Geoffrey Fox, Barbara Ann O’Leary – Indiana University

Introduction Some of the pressing needs faced by educators when teaching Cloud, Grid, HPC: Availability of suitable resources A flexible environment Fast turn-around times Reuse and sharing of educational materials FutureGrid sees education as one of its main use cases and seeks to address these needs Goals of this BOF: Highlight key aspects of FG and its use to date Benefit from your input and discussions to make it a primary resource used by you and the community for education activities 2

Needs addressed by FutureGrid Resources: FutureGrid is an XSEDE resource available to the community at large which provides unique opportunities for education Thousands of cores, multiple sites Heterogeneous: supporting HPC, cloud, Grid Flexibility: Cloud, virtualization, dynamic provisioning Environment can adapt to the user, rather than expect user to adapt to the environment 3

Needs addressed by FutureGrid Fast turn-around times Reduce barriers to entry and engage new users Web 2.0 portal to create class project and manage users; fast turn-around Reuse and sharing of educational materials Use of encapsulated “appliances” as a primary delivery mechanism of education/training modules Promoting reuse, replication, and sharing 4

Guiding principles Fidelity: activities should use full-fledged, executable software: education/training modules Learn using the proper tools Reproducibility: Creators of content should be able to install, configure, and test their modules once, and be assured of the same functional behavior regardless of where the module is deployed Incentive to invest effort in developing, testing and documenting new modules 5

Guiding principles Deployability: Students and users should be able to deploy modules in a simple manner, and in a variety of resources Reduce barriers to entry; avoid dependences upon a particular infrastructure Community-oriented: Modules should be simple to share, discover, reuse, and expand Create conditions for “viral” growth 6

FutureGrid Web Portal How classes are setup and managed Create a Web account Submit a project proposal: define a class, workshop, short course, tutorial Education is one of the key use cases of FG; eager to have more projects!!! Needs approval, but not tied to a fixed schedule Typically, turn-around of days Add users to project Users create own accounts using the portal Project leaders authorize them to gain access Students can then interactively use FG resources (e.g. to start VMs) Share and reuse – appliances, documents 7

Educational appliances A flexible, extensible platform for hands-on, lab-oriented education on FutureGrid Support clustering of resources Virtual private clusters: Virtual machines + virtual networking to create isolated, repeatable environments Virtual appliances: self-contained, pre-packaged VMs Virtual networks: simple management of virtual clusters by students and educators 8

9 Virtual appliance example Virtual appliance + virtual network copy instantiate Hadoop + Virtual Network A Hadoop worker Another Hadoop worker Repeat… Virtual machine Virtual network

Use of FutureGrid in classes Cloud/distributed computing classes U.of Florida, U. Central Florida, U. of Puerto Rico, Univ. of Piemonte Orientale (Italy), Univ. of Mostar (Croatia) Distributed scientific computing Louisiana State University Tutorials, workshops: Big Data for Science summer school A cloudy view on computing SC’11 tutorial – Clouds for science Science Cloud Summer School (July/Aug’12) 10

Cloud computing classes Massimo Canonico, U. Piemonte Orientale Difficulties to overcome: Hardware issues: find enough free physical machines able to host virtual machines Software issues: time to install/configure as many as possible different cloud platforms University was not able to provide me the necessary hardware and software support Students started to play with FutureGrid After attending few lessons, they were able to start/stop virtual instance with several Cloud Computing platforms 11

Cloud computing classes Students used Eucalyptus, OpenStack and Nimbus Half were not computer scientists. As FutureGrid freely shares their physical machines and their cloud platforms, decided to freely share all materials of my class. Hands-out, configuration files and link to useful documentation are available computing-class 12

Cloud computing classes Graduate-level “Cloud computing for Data-Intensive Sciences” (Judy Qiu, Fall 2010) Virtualization technologies and tools Infrastructure as a service Parallel programming (MPI, Hadoop) FutureGrid provided a set of software options that made it possible for students to work on different projects along the system stack. 13

Cloud Storage #8 Cloud Storage Survey (Xiaoming, Nixiaogang) Cloud Storage #8 Cloud Storage Survey (Xiaoming, Nixiaogang) Iterative MapReduce #3 LDA (Changsi, Yang) #4 MemCache (Saliya, Yiming,Jerome) #5 Avro (Yuduo, Yuan, patanachai) #6 PageRank (Shuo-Huan,Parag) Iterative MapReduce #3 LDA (Changsi, Yang) #4 MemCache (Saliya, Yiming,Jerome) #5 Avro (Yuduo, Yuan, patanachai) #6 PageRank (Shuo-Huan,Parag) Virtualization #9 Hypervisor Performance Analysis Project (James, Andrew) Virtualization #9 Hypervisor Performance Analysis Project (James, Andrew) Cloud Platform Cloud Infrastructu re Cloud Infrastructure #7 Nimbus, Eucalyptus (Stephen, Sonali, Shakeela) Cloud Infrastructure #7 Nimbus, Eucalyptus (Stephen, Sonali, Shakeela) Hypervisor/ Virtualizatio n Dryad/DryadLINQ #1 Matrix Multiplication (Swapnil,Amit,Pradnay) #2 PhyloD (Ratul,Adrija,Chengming) Dryad/DryadLINQ #1 Matrix Multiplication (Swapnil,Amit,Pradnay) #2 PhyloD (Ratul,Adrija,Chengming) Higher Level Languages Term Projects (Slide courtesy of Judy Qiu)

University of Arkansas Indiana University University of California at Los Angeles Penn State Iowa State Univ.Illinois at Chicago University of Minnesota Michigan State Notre Dame University of Texas at El Paso IBM Almaden Research Center Washington University San Diego Supercomputer Center University of Florida Johns Hopkins July 26-30, 2010 NCSA Summer School Workshop Students (200 on sites from 10 institutes; 100 online) IU MapReduce and UF Virtual Appliance technologies are supported by FutureGrid. (Slide courtesy of Judy Qiu) Big Data for Science

16 Uploading and sharing images APIs available to upload images, customize, save, and share images Community education pages are available FutureGrid Web portal allows users to publish their own content Tutorials, presentations on Web portal; VMs on image repositories

Discussion starters At a high-level, does FutureGrid provide capabilities that are aligned with your expectations for an CI useful to hosting educational activities? What would you need to evaluate whether using FutureGrid is a good idea? 17

Cyberinfrastructure What cyber-infrastructure features does FutureGrid have that may help you with educational needs? What cyber-infrastructure features may be missing which would be important? 18

Educational materials What documentation do educators, students need to use this infrastructure? What formats are effective? Tutorials, manuals, videos? 19

Topics IaaS cloud computing? Map/reduce? HPC? Grid? Applications? 20

Outreach, Community-building What information may be missing, unclear, hindering ability to reach out? Ways to attract/foster a community that will share and benefit from sharing educational materials? Within and beyond XSEDE 21

22 Questions? More information: This document was developed with support from the National Science Foundation (NSF) under Grant No to Indiana University for "FutureGrid: An Experimental, High-Performance Grid Test-bed." Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF