1. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf of P. Solagna/EGI.eu ISGC 2013, Taipei, 20 March 20131

2. www.egi.eu EGI-InSPIRE RI-261323 EGI service platforms Core platform –to enable the federation of resources Community platform –access computing resources –access and manage data and storage Cloud platform –access cloud resources ISGC 2013, Taipei, 20 March 20132

3. www.egi.eu EGI-InSPIRE RI-261323 Platforms architecture ISGC 2013, Taipei, 20 March 20133 Core platform Community platformsCloud platform Community specific services or workflows

4. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 EGI Core Platform ISGC 2013, Taipei, 20 March 2013 4

5. www.egi.eu EGI-InSPIRE RI-261323 Components EGI Core Platform –Service information Service discovery service –Service and resource status and properties (GLUE) Service registry –Accounting –Monitoring –Helpdesk ISGC 2013, Taipei, 20 March 20135

6. www.egi.eu EGI-InSPIRE RI-261323 EGI service registry: GOCDB Information provided –Topology information, production status of sites/services –Administrators contacts –Service downtimes GOCDB interfaces –Information is manually input by service administrators –Service type catalogue –GUI and Programmatic Interface http://goc.egi.eu/ ISGC 2013, Taipei, 20 March 20136

7. www.egi.eu EGI-InSPIRE RI-261323 Information discovery system BDII Distributed information system –Resource information provider –Site-BDII –Top-BDII Information available –Endpoints, capability implemented, software version –VO supported –Capacity total/available Interfaces –GLUE schema data format –LDAP for accessing the data ISGC 2013, Taipei, 20 March 20137

8. www.egi.eu EGI-InSPIRE RI-261323 Accounting Components –Sensors deployed at resorce level Harvest accounting information from local logs –Hierarchy of persistent repositories (EGI central accounting DB) –Accounting portal Standards and interfaces –Usage records (OGF) CAR and STAR will be gradually deployed in EGI during 2013CAR STAR –Secure Stomp Messenger: messaging-based (ActiveMQ) protocol to transport accounting records to the repository https://wiki.egi.eu/wiki/APEL/SSM ISGC 2013, Taipei, 20 March 20138

9. www.egi.eu EGI-InSPIRE RI-261323 Monitoring Service Availability Monitoring distributed infrastructure –NGI-level Nagios: schedules tests to check services availability –Mimic basic user behavior –ops VO MyEGI visualization tool for the monitoring results –EGI-level Metrics results store Availability and Reliability database Monitoring profiles Central MyEGI Interfaces –Test results are published to the central store using a message broker network (ActiveMQ) –Infrastructure topology information from GOCDB ISGC 2013, Taipei, 20 March 20139

10. www.egi.eu EGI-InSPIRE RI-261323 EGI Helpdesk Global Grid Users Support (GGUS). Distributed system with one central interface –User support –Operations support –Technology support Interfaces API to programmatically manage tickets Tickets assigned to specific support units can be forwarded to external (integrated) ticketing system (e.g. NGI internal helpdesk, community specific or software developers tracker) –Request Tracker already integrated ISGC 2013, Taipei, 20 March 201310

11. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 EGI Community Platforms ISGC 2013, Taipei, 20 March 2013 11

12. www.egi.eu EGI-InSPIRE RI-261323 Components Provides to the user communities the basic capabilities to access resources supporting their research activities –Access compute resources Computing element Workload management –Manage data and storage resources Storage element File catalogue –Authentication/Authorization ISGC 2013, Taipei, 20 March 201312

13. www.egi.eu EGI-InSPIRE RI-261323 Compute Element A CE provides access to the computing resources of a site –Different implementations available in EGI: CREAM, ARC-CE, Unicore/X, Globus GRAM5 –Users can submit jobs directly or indirectly to CE, monitor their status and retrieve outputs Interfaces –Not all the CE implement a common interface for job management CREAM, ARC and Unicore/X recently implemented EMI-ES Interfaces with the core platform –information discovery system to retrieve the list of resources fulfilling their requirements The CE details must be published in the information system –CEs must publish the accounting data ISGC 2013, Taipei, 20 March 201313

14. www.egi.eu EGI-InSPIRE RI-261323 Workload management Workload Management Service (WMS) acts as brokers for the computing resources –Users submit to the WMS jobs specifying user requirements –EMI-WMS retrieves the list of CEs compatible with the job requirements from the information discovery system Jobs are submitted to the CEs that fit with the requirement and with lower workload –Users can monitor their jobs and retrieve the outputs directly through the WMS –WMS can submit to CREAM, ARC-CE and GRAM5 Interfaces with the core platform –use the information discovery system to retrieve the list of CEs and their capabilities ISGC 2013, Taipei, 20 March 201314

15. www.egi.eu EGI-InSPIRE RI-261323 Storage Element Data and storage management –Store input/output data for compute jobs –File catalogue to link one or more physical file replicas to a logical file name –Data transfer and replication  File Transfer Service Interfaces –Storage Resource Management (OGF) DPM, dCache, StoRM –GridFTP file transfer, WebDAV emerging Interfaces with the core platform –Storage Element properties and status are published into the information discovery system –Storage Elements capable of producing storage accounting information (STAR), SSM for publication ISGC 2013, Taipei, 20 March 201315

16. www.egi.eu EGI-InSPIRE RI-261323 AuthN and AuthZ User authentication based on x509 certificates –Certification Authorities approved by EUGridPMA/IGTF –Towards federated identity provisioning User authorization is based on VO membership –VO membership information is provided by VOMS service Interfaces –EGI platforms services must support x509 certificates –Services supporting VOs  VOMS ISGC 2013, Taipei, 20 March 201316

17. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 Cloud Platform ISGC 2013, Taipei, 20 March 2013 17

18. www.egi.eu EGI-InSPIRE RI-261323 EGI cloud platform (in progress) Blueprint for the federation of Cloud services across EGI –Testbed of 16 resource providers, ~1000 cores –Demos presented during the main EGI events and other conferences –Purpose: integrate cloud middleware (e.g. Openstack/ OpenNebula) with the EGI core platform Accounting Monitoring Information discovery system Interfaces –OCCI to manage the virtual machines –CDMI to manage and access cloud storage –x509 for user authentication –Users can also use other management interfaces or authentication mechanisms provided by the cloud middleware ISGC 2013, Taipei, 20 March 201318

19. www.egi.eu EGI-InSPIRE RI-261323 Conclusions Federation of EGI resources and services is supported by the Core Platform –Information discovery system, service availability monitoring, accounting and helpdesk Community platforms for access to federated resources –Rely on interfaces and protocols of the core platform The Core Platform can be extended to support different type of federations –Federated cloud platform ISGC 2013, Taipei, 20 March 201319