Instituto de Biocomputación y Física de Sistemas Complejos Cloud resources and BIFI activities in JRA2 Reunión JRU Española en EGI-ENGAGE Rubén Vallés & Carlos Gimeno

Index AraGrid Cloud EPSH Cloud EUPT PireGrid Monitoring accounting and security EGI-InSPIRE and FedCloud BIFI Cloud Resources and activities EGI-ENGAGE JRA2.2 FedCloud activities

AraGrid Regional grid ( BIFI, EPSH, EUPT) Geographically distributed but managed completely by BIFI sysadmins Middleware gLite UMD SO Scientific Linux 5/6 ~ 2000 computing cores ~ 200TB RAW distributed storage 2GB RAM/core ~4TB RAM total Currently only ~ 1000 cores in grid. They will soon be migrated to cloud

Cloud EPSH Grizzly + Ceph Services: Intel XEON X5650, 6 cores, 32GB RAM Compute nodes: 36 compute nodes 2 Intel XEON X5650 processors, 6 cores each 432 cores in total 24GB of RAM Each compute node has its own nova-network service Multihost deployment CloudSME project Testbed since July 2013 and FedCloud EGI since January 2014

Cloud EUPT Havana+ Ceph Services: Intel XEON X5650, 6 cores, 32GB RAM Compute nodes: 36 compute nodes 2 Intel XEON X5650 processors, 6 cores each 432 cores in total 24GB of RAM Neutron network service Each new user can manage each own networks without collision

Cloud EUPT II

Cloud PireGrid Folsom + SWIFT DEPRECATED versión. MUST BE UPGRADED Services: Intel XEON X5620, 4 cores, 32GB RAM Compute nodes: 32 compute nodes 2 Intel XEON X5620 processors, 4 cores each 256 cores in total 16GB of RAM NAT using Nova-network installed separately In production since March 2013, previously Essex since 2012 SCI-BUS Hadoop MapReduce Infrastructure.

Monitoring, accounting and security tools Such complex infrastructures have to be monitored from the very hardware low level to the top service. Nagios, ganglia, Zabbix

CPU load

Network traffic

Ad-hoc checks

Security: logstash+kibana+elastic search

EGI-InSPIRE / FedCloud Tasks in FedCloud during 2014 Federation of european clouds using open standards. BIFI first Resource Provider to be fully VALIDATED in 2014 Testing integration of OpenStack middlewares for FedCloud Support to different user communities Development of glancepush and vmcatcher handler for OpenStack

EGI-InSPIRE / FedCloud Tasks in FedCloud during 2014 Federation of european clouds using open standards. BIFI first Resource Provider to be fully VALIDATED in 2014 Testing integration of OpenStack middlewares for FedCloud Support to different user communities Development of glancepush and vmcatcher handler for OpenStack

BIFI Cloud Resources Two cloud infrastructures under the same BIFI site Grid resources being decommissioned → cloud ~ 1000 cores more

BIFI Cloud Resources Certification status

Testing integration of OpenStack Keystone-VOMS Caso tf:WorkGroups:Scenario4#cASO Subproxies voms/tree/subproxy_support

Software development OpenStack integration with vmcatcher: OpenStack-handler-for-vmcatcher – for-vmcatcher Python-glancepush –

Support to user communities Support to developments in FedCloud: – DIRAC ( Victor Mendez ) – WS-PGRADE – CESGA tests with Hadoop VOs configured: – FedCloud VO – LHCb VO – ATLAS VO

ATLAS FedCloud: Cluster in the cloud ATLAS VO has deployed its own virtualized computing cluster at EGI FedCloud

Accounting 2014

Accounting 2015 Since 2015 ATLAS VO is using great number of resources at BIFI ( VMs )

WP4 (JRA2) Platforms for the Data Commons. Task JRA2.2 – Federated Cloud Extend AppDB The EGI Applications Database (AppDB) will evolve from its current role as catalogue of applications and virtual machines (VM) to include a graphical user interface allowing authorised users to perform basic VM management operations. Extending VM management standards support This activity will design and define two new OCCI interface extensions supporting new capabilities requested from EGI-Engage’s CC: (1) Support for users creating snapshots of running VM instances and make these available as first-class VM images that can be instantiated, and (2) support for changing attached resources to an executing VM instance.

Task JRA2.2 – Federated Cloud Relocating VM instances between providers This activity will design the workflow and interactions that are necessary to support relocating suspended VM instances from one federated cloud resource provider to another. Integration support for CMFs Participants in this activity will contribute the necessary backend (i.e. CMF) implementations of the capabilities that are developed in this task, as well as on- going maintenance support for new and existing features for future releases of OpenNebula and OpenStack deployed in the EGI Federated Cloud infrastructure.

Questions? Raise your hand…