1. 1 Grid Computing Middleware Presenter: Xon Xay Nguyen Tuan Viet Tram The Phien

2. 2 Outline Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

3. 3 Outline Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

4. 4 Grid environment

5. 5 Introduction Three main issues to confont in Grid environment: Heterogeneity – multiplicity of resources Scalability Adaptability - Resource managers or applications dynamic to extract the maximum performance from the available resources and services.

6. 6 Outline Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

7. 7 Grid Middleware System software between applications and operating system Provide services to application Discovery, storage, execution, information, service integration, resource monitoring, failure detection and recovery,… Hide heterogeneous of the Grid environment Provide standardised interfaces to services.

8. 8 Outline Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

9. 9 Purposes of Middleware Open, general-purpose and standard Standard protocols Defines the contents and sequence of message exchanges used to request remote operation Important and essential to achieve the interoperability that Grid depends on Standard APIs Interfaces to code libraries Facilitate construction of Grid components by allowing code components to be reused

10. 10 Outline Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

11. 11 Grid layered architecture - The neck: a small set of core abstractions and protocols - Top of hourglass: many different high- level behaviors can be mapped - Base of hourglass: can be mapped onto many different underlying technologies

12. 12 Component of Middleware Connectivity layer Resource layer Collective layer

13. 13 Connectivity layer Communicating easily and securely Communication protocol: exchange data Authentication protocol: cryptographically secure mechanism for verifying the identity of users and resources. Single sign-on Delegation Integration with various local security solutions User-based trust relationships

14. 14 Resource layer Sharing single resources Define protocols (and APIs and SDKs) for the secure negotiation, initiation, monitoring, control, accounting, and payment of sharing operations on individual resources Information protocols - obtain information about the structure and state of a resource Management protocols - negotiate access to a shared resource

15. 15 Collective layer Coordinating multiple resources Implement a wide variety of sharing behaviors without placing new requirements on the resources being shared Example – Directory services, co-allocation, scheduling, brokering services, Monitoring and diagnostics services, Data replication services, Grid-enabled programming systems, …

16. 16 Collective layer- Example Application use co- reservation service protocol to request end- to-end network reservation We define a co- reservation service protocol Implement co- reservation service Call co-allocation API to implement co-allocation operation

17. 17 Outline Introduction Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

18. 18 Globus Toolkit Evolution Globus Toolkit version 2 Globus Toolkit version 3 Globus Toolkit version 4

19. 19 Globus evolution 1997 GT2.0 2003 GT3.0 OGSA 2005 GT4.x WSRF Proof of conceptStandardization

20. 20 Globus Toolkit version 2 Community-based, open-architecture, open source set of services and software libraries that support Grids and Grid applications. De facto standard for Grid computing Focus on usability and interoperability Middleware GT2 – Connectivity layer GT2 - Resource layer GT2 - Collective layer

21. 21 GT2 – Connectivity Authentication protocol Public-key based GSI protocols GSI - Grid Security Infrastructure authentication, communication protection, authorization GSI builds on and extends the TLS protocols TLS - Transport Layer Security TLS - defines a protocol to provide privacy and data integrity between two communicating applications.

22. 22 GT2 – Connectivity layer Communication protocol Internet layered protocol architecture Internet layer - IP and ICMP Transport - TCP, UDP Application layer - DNS, OSPF, RSVP, …

23. 23 GT2 - Resource layer A Grid Resource Information Protocol – base on Lightweight Directory Access Protocol (LDAP) HTTP-based GRAM protocol GRAM - Grid Resource Access and Management allocation of computational resources monitoring and control of computation on resources GridFTP - a management protocol for data access An extended version of the File Transfer Protocol

24. 24 GT2 - Collective layer build on Globus Connectivity and Resource protocols Meta Directory Service introduces Grid Information Index Servers (GIISs) LDAP information protocol Access resource-specific GRISs to obtain resource state GRRP used for resource registration. DUROC resource co-allocation library provides an SDK and API for resource co-allocation

25. 25 Globus Toolkit version 3 Build on and significantly extend GT2 concepts and technologies Base on Open Grid Services Architecture Factoring of component behaviors – identify essential Grid functions and express Service orientation – enable a uniform treatment of all network entities Align with Web services – WSDL (Web Services Definition Language)

26. 26 Globus Toolkit version 4 developed by The Globus Alliance OGSA requires “stateful services” WSRF – Web Service Resources Framework developed by OASIS (http://www.oasis-open.org) WSRF provides the stateful services that OGSA needs

27. 27 Relationship between OGSA, GT4, WSRF, and Web Services

28. 28 gLite Toolkit The Enabling Grids for E-sciencE project (EGEE) Middleware stack that combines components developed in various related projects.

29. 29 gLite middleware A layer between services and resources gLite follows a Service Oriented Architecture

30. 30 gLite - guiding principles Service oriented approach Allow for multiple interoperable implementations Lightweight (existing) services Easily and quickly deployable Use existing services where possible Condor, EDG, Globus, LCG, … Portable Being built on Scientific Linux and Windows Security Sites and Applications

31. 31 gLite - guiding principles Performance/Scalability & Resilience/Fault Tolerance Comparable to deployed infrastructure Co-existence with deployed infrastructure Co-existence with LCG-2 and OSG (US) are essential for the EGEE Grid services Site autonomy Reduce dependence on ‘global, central’ services Open source license

32. 32 gLite Architecture

33. 33 gLite - Access All of the gLite services are accessible via APIs and CLIs. API - Application Programming Interface CLI - Command Line Interface

34. 34 gLite - Security Services

35. 35 gLite - Security Services Authentication identify entities PKI (X.509) infrastructure with CA third party for signature Authorization allows or denies access to specific services three basic authorization models, classified as agent, push and pull. Dynamic Connectivity Service tackle the problems that arise when connectivity is restricted by the resource owner. Auditing monitoring and providing for post-mortem analysis of security related events.

36. 36 gLite - Infomation and Monitoring Services

37. 37 gLite - Infomation and Monitoring Services Infomation and Monitoring Services a vital low-level component of any grid

38. 38 gLite - Infomation and Monitoring Services Job Monitoring use the APIs offered by the apache logging services (log4j) Service Discovery a facility offered to both end users and to other services to locate suitable services. Network Performance Monitoring define a standard interface to monitoring frameworks. NPM Publisher for use by gLite components.

39. 39 gLite - Helper Services

40. 40 gLite - Helper Services Configuration and Instrumentation Service common, standard-based configuration and instrumentation functionality to the gLite services. Bandwidth Allocation and Reservation Service control and balance the usage of the network and categorise and prioritise traffic flows. Agreement Service implements the communication protocol used to exchange information about Service Level Agreements (SLAs) and defines the SLA structure.

41. 41 gLite - Data Services

42. 42 gLite - Data Services Catalog store information about the data and metadata Storage Element is the Grid service responsible for saving/retrieving files to/from some data store which can provide a wide range of quality of service to the user. Data movement provide scalable and robust managed data transfer between Grid sites

43. 43 gLite - Job Management Services

44. 44 gLite - Job Management Services Accounting accumulates information about the usage of Grid resources Computing element representing a computing resource job management and other capabilities Workload Management comprises a set of Grid middleware components responsible for the distribution and management of tasks across Grid resources

45. 45 gLite - Job Management Services Job Provenance keep track of the definition of submitted jobs, execution conditions and environment. Package Manager is a helper service that automates the process of installing, upgrading configuring, and removing software packages from a shared area

46. 46 Comparison Globus Toolkit - faster and more efficient gLite - sophisticated and developer friendly Document Globus documents - sketchy and missing gLite documents - sufficient, available

47. 47 Summary Definition of Grid Middleware Purpose of Middleware Grid layered architecture Technology Globus GLite

48. 48 Reference I. Foster and C. Kesselman, The Grid: Blueprint for a New Computing Infrastructure. Morgab Kaufmann Publishers, 1999. Fran Berman, Anthony J. G. Hey and Geoffrey C. Fox, Grid computing: Making the Global Infrastructure a Reality. John Wiley & Sons Ltd, 2003. Globus project: http://www.globus.org/alliance/ Grid Computing Slides – Nguyen Tuan Anh Grid Computing Slides – Pham Tran Vu vonLaszewski--grid-middleware 03-grid-concepts-architecture-middleware-adler-vortrag Websites of gLite (http://www.eu-egee.org)

49. 49 Q&A