1. Basics of Grid Middleware – 2 (with an introduction to OMII-Europe) Mike Mineter NeSC-TOE

2. EU project: RIO31844-OMII-EUROPE 2 Contents Convergence of Web Services and Grids Current state of production grids (Some of the) emerging standards Response of the OMII-Europe project Acknowledgement: many of these slides are reused from presentations created by the NeSC-TOE team for NGS courses

3. EU project: RIO31844-OMII-EUROPE 3 Goal of talk To explore the –Multiple grids and diverse middleware in production use –Emerging standards …. and the OMII-Europe response –building bridges between grids

4. EU project: RIO31844-OMII-EUROPE 4 Service orientation – software components that are… Accessible across a network Loosely coupled, defined by the messages they receive / send Service description that can be used to create client software Based on standards (for which tools do / could exist) Developed in anticipation of new uses Service Client

5. EU project: RIO31844-OMII-EUROPE 5 Using service B from service A Web services container service A: consumes B TCP/IP Messages O/S Web services container service B O/S A B messages

6. EU project: RIO31844-OMII-EUROPE 6 Using service B from service A Web services container service A: consumes B TCP/IP Messages O/S Web services container service B O/S A B messages These messages define service B

7. EU project: RIO31844-OMII-EUROPE 7 Using service B from service A Web services container service A: consumes B TCP/IP Messages O/S Web services container service B O/S A B messages http, https Commonly used for WS - original purpose: carry HTML HTTP request methods GET : specify a URL POST: key-value pairs Defaults to port 80 HTTPS: encrypts / decrypts

8. EU project: RIO31844-OMII-EUROPE 8 Messages in XML Using service B from service A Web services container service A: consumes B TCP/IP O/S Web services container service B O/S A B messages http, https XML eXtendable Markup Language Human readable (sort of) – so not tied to one architecture Extendable: can define new data types, belong to “namespaces”, expressed by schemas

9. EU project: RIO31844-OMII-EUROPE 9 Messages in XML WSDL Web services container service A: consumes B TCP/IP O/S Web services container service B O/S A B messages http, https Service B is described by WSDL, “Web Service Description Language”. Includes: Where: “ end point” URL where B receives messages What: Message definitions How: “Bindings” to use service

10. EU project: RIO31844-OMII-EUROPE 10 Messages in XML WSDL Web services container service A: consumes B TCP/IP O/S Web services container service B O/S A B messages http, https Service B is described by WSDL, “Web Service Description Language”. Includes: Where: “ end point” URL where B receives messages What: Message definitions How: “Bindings” to use service API’s for B API’s are derived from the WSDL by tools Developer adds code for specific client NO API LIBRARY SUPPLIED

11. EU project: RIO31844-OMII-EUROPE 11 SOAP Web services container service A: consumes B TCP/IP O/S Web services container service B O/S A B messages http, https API’s for B XML-based messaging protocol Header: extendable for security, accounting,.. Body: message content

12. EU project: RIO31844-OMII-EUROPE 12 SOAP Code languages Web services container service A: consumes B TCP/IP O/S Web services container service B O/S A B messages http, https API’s for B This determines whether code is in Java, C, Perl, Python…

13. Enabling Grids for E-sciencE INFSO-RI-508833 13 HTTP message (JAVA) Web Services A B.Op (a,b) invoke c return B Insert the web into the invoke and into the return Consumes B (client) Stack Sk eleton Provides B (server) Ret(c) Soap envelope A B- stub Stack B Bs.Op (a,b) invoke … … B.Op (a,b) invoke … c return c … Web Soap envelope HTTP message Op(a,b) WSDL for B Soap envelope Java program with 2 classes

14. Enabling Grids for E-sciencE INFSO-RI-508833 14 HTTP message (JAVA) Web Services A B.Op (a,b) invoke c return B Insert the web into the invoke and into the return Consumes B (client) Stack Sk eleton Provides B (server) Ret(c) Soap envelope A B- stub Stack B Bs.Op (a,b) invoke … … B.Op (a,b) invoke … c return c … Web Soap envelope HTTP message Op(a,b) WSDL for B Soap envelope Java program with 2 classes

15. 15 Real Web Service Invocation Borja Sotomayor, http://gdp.globus.org/gt4-tutorial/multiplehtml/ch01s02.html Discover Describe Invoke

16. Enabling Grids for E-sciencE INFSO-RI-508833 16 WS-I core of Web Services WS-I (Interoperability) delivers practical guidance, best practices and resources for developing interoperable Web services solutions. http://www.ws-i.org/ Open standards: SOAP: protocol for message passing Web Service Description Language: to describe services UDDI: Universal Description, Discovery and Integration WS-Security: incorporates security

17. EU project: RIO31844-OMII-EUROPE 17 Web Services Grid Technology Grid Services Commerce- driven Standards Tools Research driven Data-intensive Compute intensive Collaboration – sharing of resources October 2001 View Open Grid Services Architecture

18. EU project: RIO31844-OMII-EUROPE 18 2007 View WSRF – Web Services Resource Framework WS-Notification Web Services –Basis for defining standards for different services –For services on grids: Need to manage state – interact with resources Need to be notified of change of state

19. © 2004 IBM Corporation © 2004 University of Chicago 19 context Interface Web Service message id message Using a Web service to access a WS-Resource id address resource Endpoint Reference The WS-Resource framework model Run-time environment

20. © 2004 IBM Corporation © 2004 University of Chicago 20  WS-Resource Properties Resource state and metadata “Projected” as an XML document Query and Set operations  WS-Resource LifeTime Explicit destruction or “Soft state” time-to-live Provides for cleanup of resource instances resource 5A34C1DE03 Power6.2 400 256 1 The WS-Resource framework model

21. Enabling Grids for E-sciencE INFSO-RI-508833 21 Acknowledgements: some slides are from http://www.nesc.ac.uk/action/esi/contribution.cfm?Title=385

22. EU project: RIO31844-OMII-EUROPE 22 Among middleware that uses WSRF/ WS-N: –Globus Toolkit 4 –UNICORE

23. 23 Custom Web Services WS-Addressing, WSRF, WS-Notification Custom WSRF Web Services GT4 WSRF Web Services WSDL, SOAP, WS-Security User Applications Registry Administration GT4 Container Globus Toolkit 4 Web Services Core Thanks to J. Schopf, ANL

24. EU project: RIO31844-OMII-EUROPE 24 Supporting the Perspective: “Driving HPC in the pan-European ecosystem“ –Leadership capability computing (tier-0)  “Supercomputers” –Entry-level capability computing (tier-1)  “Clusters” –Farming-based capacity computing (tier-2)  “PC pools, farms” Partnership for Advanced Computing in Europe (PACE) –Towards multi-core petascale Supercomputing Grids –Near Future: Multi-core-based supercomputers (‘e.g. 80 cpus on a chip’) Up to 1 Mil. CPUs at 1 site for each supercomputer Grid: 1 Mil * n CPUs „Grid driving HPC“ Thanks to M. Riedel, FZJ

25. EU project: RIO31844-OMII-EUROPE 25 WSRF-compliant and OGSA-based UNICORE 6 –Standards: WSRF 1.2 final, WS-I, JSDL 1.0, XACML 1.0, OGSA ByteIO –Modern software stack: Java 5, XFire SOAP Stack, XMLBeans, Jetty, … Joint development effort under leadership of FZJ Beta version released in April 2007 6.0 final release, July/August 2007, rc already available –UNICORE Atomic Services (UAS), workflows, compliant with UNICORE 5 TSIs, Intel GPE 1.4, UCC 6.1 release, Q4-2007 –Extended workflow support, portals, Intel GPE 1.5 –support for VOMS and OGSA-BES (out of OMII-Europe) Standards-based UNICORE 6 Development Thanks to M. Riedel, FZJ

26. EU project: RIO31844-OMII-EUROPE 26 Contents Convergence of Web Services and Grids Current state of production grids (Some of the) emerging standards Response of the OMII-Europe project

27. EU project: RIO31844-OMII-EUROPE 27 Grid Islands gLiteGlobus Toolkit 4 UNICORE CROWN Isolate: Data Computers Expertise

28. EU project: RIO31844-OMII-EUROPE 28 Bridges in infrastructure... What do they enable?

29. EU project: RIO31844-OMII-EUROPE 29 How are they built?

30. EU project: RIO31844-OMII-EUROPE 30 How easy are they to use?

31. EU project: RIO31844-OMII-EUROPE 31 Overview of OMII-Europe Bridge-building with OMII-Europe –What do these bridges enable? –How are they built? –How easy are they to use? –When will they be ready to use? Most release dates after ~September. Project is only just 1 year old! Happy Birthday to us!

32. EU project: RIO31844-OMII-EUROPE 32 Contents Convergence of Web Services and Grids Current state of production grids Emerging standards Response of the OMII-Europe project

33. EU project: RIO31844-OMII-EUROPE 33 Grid Islands gLiteGlobus Toolkit 4 UNICORE CROWN Isolate: Data Computers Expertise

34. EU project: RIO31844-OMII-EUROPE 34 OMII-Europe vision: to enable… gLiteGlobus Toolkit 4 UNICORE CROWN “The global grid” VO to span grids Application portability

35. EU project: RIO31844-OMII-EUROPE 35 Bridges – for interoperability – initially…. gLiteGlobus Toolkit 4 UNICORE CROWN Job execution Data access and integration Accounting

36. EU project: RIO31844-OMII-EUROPE 36 Common security base Bridges – How are they built? Components based on standards

37. EU project: RIO31844-OMII-EUROPE 37 Bridges – How are they built? Components based on standards: OGSA Basic Execution Service OGSA Data Access & Integration Services OGSA Resource Usage Service OGSA: Open Grid Services Architecture

38. EU project: RIO31844-OMII-EUROPE 38 OMII-Europe components Basic Execution Service –Execution of job described in JSDL: “Job Submission Description Language” –E.g. OGSA-BES interface being developed for EGEE Compute Elements.

39. London e-Science Centre 39 Hello World in JSDL <jsdl:JobDefinition xmlns:jsdl=“http://schemas.ggf.org/2005/11/jsdl”http://schemas.ggf.org/2005/11/jsdl” xmlns:jsdl-posix= “http://schemas.ggf.org/jsdl/2005/11/jsdl-posix”>http://schemas.ggf.org/jsdl/2005/11/jsdl-posix /bin/echo hello world <jsdl:JobDefinition xmlns:jsdl=“http://schemas.ggf.org/2005/11/jsdl”http://schemas.ggf.org/2005/11/jsdl” xmlns:jsdl-posix= “http://schemas.ggf.org/jsdl/2005/11/jsdl-posix”>http://schemas.ggf.org/jsdl/2005/11/jsdl-posix /bin/echo hello world Reused with permission – slide from Stephen McGough

40. EU project: RIO31844-OMII-EUROPE 40 OMII-Europe components Basic Execution Service Data Access and Integration –Expose data to grid users –“Activities” support computation close to data –OMII-Europe porting OGSA-DAI to gLite, UNICORE, CROWN Resource Usage Service –Gather accounting data from diverse resources on different grids

41. EU project: RIO31844-OMII-EUROPE 41 X.509 VOMS +… Common security base Bridges – How are they built?

42. EU project: RIO31844-OMII-EUROPE 42 VOMS in OMII-Europe VOMS: to manage VO membership as basis for Authorisation –Used by gLite: communicate VO attributes in proxy extensions Permits delegation In OMII-Europe: also will communicate in “Security Assertion Markup Language (SAML)” –standard from OASIS

43. EU project: RIO31844-OMII-EUROPE 43 Common security base Components – How easy are they to use? Components based on standards

44. EU project: RIO31844-OMII-EUROPE 44 You will easily be able to find out! Evaluation infrastructures Common security base Components based on standards OMII-Europe Gateway (portal) Components – How easy are they to use?

45. EU project: RIO31844-OMII-EUROPE 45 Evaluation Infrastructures Ready for you to try different middleware New OMII-Europe components will be installed on these http://support.omii- europe.orghttp://support.omii- europe.org MiddlewareSite Globus Toolkit 4.0.3 Edinburgh UNICORE 5UNICORE 5, UNICORE 6 UNICORE 6 FZJ gLite 3.0 gLite 3.0 (EGEE) PSNC gLite 3.1 gLite 3.1 (EGEE) INFN OMII-UK Release 3.2 SOTON CROWN GridBU and TU

46. EU project: RIO31844-OMII-EUROPE 46 What will OMII-Europe deliver? Repository of open-source, quality assured software services for EGEE, Globus, UNICORE and CROWNgrid –Objective: Some services bundled with major grid distributions –Initial integration work with EGEE, UNICORE and Globus Evaluation infrastructure to “test” services User support and training

47. EU project: RIO31844-OMII-EUROPE 47 OMII-Europe: Summary Standards-based components Repository Evaluation infrastructures Training Support New e-science Grid Islands International, (sub-)national, community Dissemination

48. EU project: RIO31844-OMII-EUROPE 48 Further Information WS-I –http://en.wikipedia.org/wiki/WS-I_Basic_Profilehttp://en.wikipedia.org/wiki/WS-I_Basic_Profile –http://en.wikipedia.org/wiki/WS-Securityhttp://en.wikipedia.org/wiki/WS-Security Globus Toolkit: –Tutorial later today…. –http://www.globus.org/wsrf/http://www.globus.org/wsrf/ –http://gdp.globus.org/gt4-tutorial/http://gdp.globus.org/gt4-tutorial/ –http://dev.globus.org/wiki/Outreach/Materialshttp://dev.globus.org/wiki/Outreach/Materials UNICORE –Tutorial tomorrow…. –www.unicore.euwww.unicore.eu

49. EU project: RIO31844-OMII-EUROPE 49 Thank you! http://omii-europe.org