EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Services for Distributed e-Infrastructure Access Tiziana Ferrari on behalf of P. Solagna/EGI.eu ISGC 2013, Taipei, 20 March 20131

EGI-InSPIRE RI EGI service platforms Core platform –to enable the federation of resources Community platform –access computing resources –access and manage data and storage Cloud platform –access cloud resources ISGC 2013, Taipei, 20 March 20132

EGI-InSPIRE RI Platforms architecture ISGC 2013, Taipei, 20 March Core platform Community platformsCloud platform Community specific services or workflows

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Core Platform ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI Components EGI Core Platform –Service information Service discovery service –Service and resource status and properties (GLUE) Service registry –Accounting –Monitoring –Helpdesk ISGC 2013, Taipei, 20 March 20135

EGI-InSPIRE RI EGI service registry: GOCDB Information provided –Topology information, production status of sites/services –Administrators contacts –Service downtimes GOCDB interfaces –Information is manually input by service administrators –Service type catalogue –GUI and Programmatic Interface ISGC 2013, Taipei, 20 March 20136

EGI-InSPIRE RI Information discovery system BDII Distributed information system –Resource information provider –Site-BDII –Top-BDII Information available –Endpoints, capability implemented, software version –VO supported –Capacity total/available Interfaces –GLUE schema data format –LDAP for accessing the data ISGC 2013, Taipei, 20 March 20137

EGI-InSPIRE RI Accounting Components –Sensors deployed at resorce level Harvest accounting information from local logs –Hierarchy of persistent repositories (EGI central accounting DB) –Accounting portal Standards and interfaces –Usage records (OGF) CAR and STAR will be gradually deployed in EGI during 2013CAR STAR –Secure Stomp Messenger: messaging-based (ActiveMQ) protocol to transport accounting records to the repository ISGC 2013, Taipei, 20 March 20138

EGI-InSPIRE RI Monitoring Service Availability Monitoring distributed infrastructure –NGI-level Nagios: schedules tests to check services availability –Mimic basic user behavior –ops VO MyEGI visualization tool for the monitoring results –EGI-level Metrics results store Availability and Reliability database Monitoring profiles Central MyEGI Interfaces –Test results are published to the central store using a message broker network (ActiveMQ) –Infrastructure topology information from GOCDB ISGC 2013, Taipei, 20 March 20139

EGI-InSPIRE RI EGI Helpdesk Global Grid Users Support (GGUS). Distributed system with one central interface –User support –Operations support –Technology support Interfaces API to programmatically manage tickets Tickets assigned to specific support units can be forwarded to external (integrated) ticketing system (e.g. NGI internal helpdesk, community specific or software developers tracker) –Request Tracker already integrated ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI EGI Community Platforms ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI Components Provides to the user communities the basic capabilities to access resources supporting their research activities –Access compute resources Computing element Workload management –Manage data and storage resources Storage element File catalogue –Authentication/Authorization ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI Compute Element A CE provides access to the computing resources of a site –Different implementations available in EGI: CREAM, ARC-CE, Unicore/X, Globus GRAM5 –Users can submit jobs directly or indirectly to CE, monitor their status and retrieve outputs Interfaces –Not all the CE implement a common interface for job management CREAM, ARC and Unicore/X recently implemented EMI-ES Interfaces with the core platform –information discovery system to retrieve the list of resources fulfilling their requirements The CE details must be published in the information system –CEs must publish the accounting data ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI Workload management Workload Management Service (WMS) acts as brokers for the computing resources –Users submit to the WMS jobs specifying user requirements –EMI-WMS retrieves the list of CEs compatible with the job requirements from the information discovery system Jobs are submitted to the CEs that fit with the requirement and with lower workload –Users can monitor their jobs and retrieve the outputs directly through the WMS –WMS can submit to CREAM, ARC-CE and GRAM5 Interfaces with the core platform –use the information discovery system to retrieve the list of CEs and their capabilities ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI Storage Element Data and storage management –Store input/output data for compute jobs –File catalogue to link one or more physical file replicas to a logical file name –Data transfer and replication  File Transfer Service Interfaces –Storage Resource Management (OGF) DPM, dCache, StoRM –GridFTP file transfer, WebDAV emerging Interfaces with the core platform –Storage Element properties and status are published into the information discovery system –Storage Elements capable of producing storage accounting information (STAR), SSM for publication ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI AuthN and AuthZ User authentication based on x509 certificates –Certification Authorities approved by EUGridPMA/IGTF –Towards federated identity provisioning User authorization is based on VO membership –VO membership information is provided by VOMS service Interfaces –EGI platforms services must support x509 certificates –Services supporting VOs  VOMS ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Cloud Platform ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI EGI cloud platform (in progress) Blueprint for the federation of Cloud services across EGI –Testbed of 16 resource providers, ~1000 cores –Demos presented during the main EGI events and other conferences –Purpose: integrate cloud middleware (e.g. Openstack/ OpenNebula) with the EGI core platform Accounting Monitoring Information discovery system Interfaces –OCCI to manage the virtual machines –CDMI to manage and access cloud storage –x509 for user authentication –Users can also use other management interfaces or authentication mechanisms provided by the cloud middleware ISGC 2013, Taipei, 20 March

EGI-InSPIRE RI Conclusions Federation of EGI resources and services is supported by the Core Platform –Information discovery system, service availability monitoring, accounting and helpdesk Community platforms for access to federated resources –Rely on interfaces and protocols of the core platform The Core Platform can be extended to support different type of federations –Federated cloud platform ISGC 2013, Taipei, 20 March