The future of Globus (Grid meets Cloud) Ian Foster Computation Institute University of Chicago & Argonne National Laboratory

2 UC/ANL/ISI perspective on Globus l Globus is a community of contributors l This presentation is focused on UC/ANL/ISI planned contributions l Globus UC/ANL u NSF CDIGS: development & support u DOE CEDPS: Globus.org data grid R&D u caBIG/caGrid: for biomedical applications u Many others…

3 UC/ANL leadership and structure l Steve Tuecke resuming day-to-day technical and project leadership l Product teams u Adopted Agile scrum methods u Jira.globus.org with Greenhopper l User services u Consulting, operations and support u Led by Paul Dave

Grid = federation Cloud = hosting

5 Anatomy of the Grid Application Fabric Controlling things locally: Access to, and control of resources Connectivity Talking to things: communication (Internet protocols) and security Resource Sharing single resources: negotiating access, controlling use Collective Coordinating multiple resources: ubiquitous infrastructure services, app-specific distributed services Internet Transport Application Link Internet Protocol Architecture The Anatomy of the Grid: Enabling Scalable Virtual Organizations, Foster, Kesselman, Tuecke, Intl Journal of High Performance Computing Applications, 15(3), 2001.

6 Cloud Taxonomy Software-as-a-Service (SaaS) Platform-as-a-Service (PaaS) Infrastructure-as-a-Service (IaaS)

7 Some strengths of Globus l Grid Security Infrastructure ecosystem l GridFTP ecosystem l GT4 Java Core use by caGrid l Strong community

8 Some issues Facing Globus l Aging technology base u Apache Axis 1.x, PureTLS l Sustainability l Limited adoption of some components u MDS, RFT l Confusing duplication u GRAM2 vs GRAM4 l Limited collective layer, end-to-end functionality

9 Globus Toolkit Assortment of Components for Grid Builders l Focus on Connectivity and Resource layers u GRAM, GSI-OpenSSH: Run programs u GridFTP: Access file systems u OGSA-DAI, caGrid: Access databases u GSI, Myproxy, GAARDS: Security u XIO, Java Core, C Core: Communication l A few simple Collective layer components u RLS: Replica tracking u RFT: Reliable file transfer

10 GRAM5 l Now you can have reliability and scalability l Backward compatible with GRAM2 u 2 minor exceptions l Job rendezvous for MPICH-G, stdout/err streaming u Tested w/ GRAM4 Java & C clients, Condor-G l Improved functionality u Exit codes u Better logging u Enhanced metrics u Etc.

11 Reliable File Transfer (RFT) l Replaced by Globus.org

12 MDS l Mostly used for service registries/catalogs u Starting new IIS effort with this focus l Better tools (e.g. Nagios) for monitoring

13 Other Components Continuing l GridFTP l RLS l Myproxy l GSI-OpenSSH l GAARDS l Introduce l OpenNebula l Nimbus l …

14 Metrics l New metrics collector u More reliable and scalable l New metrics generators u GRAM5 u Myproxy u Others coming

15 Globus Toolkit Roadmap l GT 5.0: 4Q2009 u GridFTP, GRAM5, RSL, Myproxy, GSI-OpenSSH u C libraries: GSSAPI, XIO, C Core, etc u jGlobus: Security, GRAM client, GridFTP client l GT 5.2: 2Q2010 u Focus: Native Packaging l GT 5.4: 4Q2010 u Focus: Usability l GT 4.x maintenance and support will continue at least through end of 2010

16 Crux Java SOA for Grid Applications l Core: Apache CXF + Globus security l Also: Introduce + OSGi + ServiceMix ESB l WSRF subset compatibility supported l Taverna workflows l Replacement for GT4 Java Core u Programming model not the same

17 Many custom Collective layers l Most Grid deployments have custom, domain-specific Collective layer u Built on common Resource layer components u E.g. Data transfer and mirroring, workflows, … l Challenges: u Expensive to develop u Expensive to operating and supporting u Useful for narrow community l How do we make these capabilities available to more users?

18 Globus.org Service Collective Layer Grid Services via Cloud SaaS l End-to-end collective layer functionality targeted toward end users u Generalize lessons from custom Grids u Focus on ease of use, federation l Hosted and supported by Globus team l Initial focus on file transfer u Near term: Add sync, mirroring, caching u Long term: Add job execution, workflows, VO management

19 Applying SaaS Techniques l Service: Built as scale-out web application u Hosted on Amazon Web Services l Client: Minimize software deployment u Web 2.0 l AJAX + REST l Notification via , IM, SMS, Twitter, etc. l Enable mash-ups u CLI 2.0 l ssh cli.globus.org … u Dynamic deployment of resource integrators u Mobile clients

20 Web 2.0 GUI

21 iPhone Client

22 Upcoming Events l GlobusWorld: March Argonne

Globus governance

24 Governance issues l Who decides what is called Globus? l How do we encourage contributions? l How do we encourage the development of a healthy ecosystem of producers and consumers? l How do we achieve sustainability?

25 Governance history l First, Globus Project u Argonne and USC/ISI l Then, Globus Alliance u NCSA, Edinburgh, U.Chicago u Commitment to cooperate l And now, dev.globus u Apache-like governance structure u Globus Management Committee u Dev.globus incubator process