E-science grid facility for Europe and Latin America JRA1 Status Report Development of Services for Applications and Infrastructure Francisco Brasileiro (JRA1 Activity Manager) UFCG (Brazil) First Project Review Brussels,

Brussels, First Project Review, Outline Activity goals Work planned for the first year Results achieved Highlights Work planned for the second year

Brussels, First Project Review, Activity Goals (1/2) In the DoW the JRA1 goal is stated as: “… the EELA-2 Joint Research Activity will focus on extending the versatility of the production service managed by SA1 and functionalities as required by the NA3 applications running on it. In particular, research and development will be conducted with the goal of increasing the reach and the usability of e- Infrastructure by assembling/re-engineering existing technologies and developing new ones that will facilitate the installation, management and use of the grid infrastructure. Special attention will be given to the issues related with the sustainable use, management and growth of the e- Infrastructure and its users’ community.”

Brussels, First Project Review, Activity Goals (2/2) Making the e-Infrastructure more useful and wide spread to increase its reach …  Execution of grid middleware on top of different platforms, including Microsoft Windows  Scavenging of idle resources, particularly in shared desktops … and its usability  New application-oriented grid services  Support for the integration of a lightweight opportunistic grid middleware (OurGrid) whose development is led by one of the consortium members With the effect of helping in fostering the sustainability of the e-Infrastructure

Brussels, First Project Review, Work Planned for the 1 st Year MJRA1.1: PKI-enabled OurGrid middleware (M06) –Allow the OurGrid middleware to use a Public Key Infrastructure (PKI) for authentication/authorization, as done by gLite (interoperability) MJRA1.2: Prototypes of the proposed services available at the EELA-2 software repository (M10) –The proposed application-oriented services were chosen based on the anticipated requirements of the legacy applications from the phase 1 and on the new proposed applications –The proposed infrastructure-oriented services aim at addressing the growth of the e-infrastructure as well as the simplification of its management MJRA1.3: Common Information System between gLite and OurGrid (M10) –Provide the necessary mechanism for the OurGrid and gLite Information Systems to be combined into a single Information System (interoperability)

Brussels, First Project Review, Results Achieved

Brussels, First Project Review, EELA-2 Software Repository

Brussels, First Project Review, PKI-enabled OurGrid middleware (MJRA1.1) X.509-enabled OurGrid middleware available –Enables an OurGrid system to use the same PKI and Grid Security Infrastructure technologies used by gLite systems –Users of an OurGrid system can be authenticated using the same types of certificates that they use in a gLite system –Allows the use of the standard VOMS protocol for authorisation As a result: –Users/resources of the EELA-2 e-infrastructure will require a single certificate no matter which middleware they are using/running –A few resource centres in the EELA-2 infrastructure are currently using this software

Brussels, First Project Review, Prototypes of the application-oriented proposed services (MJRA1.2) Data Management –Digital archives –Secure storage –Cooperative annotation of data Job management –VO compliant Virtual Machine environments –Workflow for OurGrid jobs

Brussels, First Project Review, Job management –WatchDog  Allows the monitoring and control of job execution on the gLite Worker Node Catalogue and file management –lgc-rec toolkit  Implements a recursive version of the lgc-* suite of commands Prototypes of the application-oriented additional services (MJRA1.2)

Brussels, First Project Review, Port of gLite to Microsoft Windows platforms –User Interface  Grid Security Infrastructure  Workload Management  File Transfer  Data Management  Information System –Computing Element  The farm is composed by a unique central node with Linux operating system in which the Gatekeeper and the Torque/MAUI head node are running  A set of worker nodes with our rebuilt version of the gLite WN package implement the farm workers Prototypes of the infrastructure-oriented proposed services (MJRA1.2)

Brussels, First Project Review, Support for grid operation –SAGE (Storage Accounting for Grid Environments) –Multiple middleware resource centres  gLite and Globus Toolkit 4 (GT4) User Interface having clients for both gLite and GT4 A single cluster running both a gLite and a GT4 Computing Element  Co-existence of gLite and OurGrid Scavenging of idle cycles in dedicated resources running gLite Increased support for Bag-of-Tasks (BoT) applications through the OurGrid middleware Prototypes of the infrastructure-oriented additional services (MJRA1.2)

Brussels, First Project Review, This milestone was foreseen as an important step towards the interoperability between gLite and OurGrid However, during the course of the development we concluded that this integration was no longer required –We will be able to leverage efforts carried out by the EDGeS project that have already addressed this issue –The EDGeS G3 (Generic Grid Gateway) technology will allow the development of gateways between gLite and OurGrid by simply providing appropriate plug-ins for the OurGrid middleware (gLite plug-ins are already available) The saved time allowed us to explore other middleware integration approaches, such as the co-existence previously mentioned Common Information System (MJRA1.3)

Brussels, First Project Review, Highlights of the 1 st Year (1/3) A number of applications are either using or being upgraded to use the services developed ServiceApplications GSAFAîuri – UFRJ (Brazil) OurGridDistBlast - UnB (Brazil), BRAMS - UFCG (Brazil) and Cinefilia - INFN (Italy) watchdogCROSS-Fire – U. of Minho (Portugal) lcg-rec toolkit CROSS-Fire – U. of Minho (Portugal) ServiceApplications GSAFgRREEMM - Cubaenergia (Cuba) Secure Storage CardioGrid Portal - UBA (Argentina) watchdogG-InterproScan and G-HMMER - UNIANDES (Colombia) lcg-rec toolkit META-Dock - UNAM (Mexico) Using Being upgraded

Brussels, First Project Review, A number of resource centres are using or starting to deploy the OurGrid middleware –Resource centres using OurGrid –Resource centres that have started the deployment  UnB and UNILASALLE (Brazil), UNIANDES (Colombia), PUCP and USMP (Peru) Highlights of the 1 st Year (2/3)

Brussels, First Project Review, Highlights of the 1 st Year (3/3) Interoperability between OurGrid and gLite planned for M16 is advancing very well –MoU signed with other projects working on similar goals (EDGeS and SEE-GRID-SCI) –Initial assessment of the possibility of using EDGeS G3 indicates the feasibility of using technology that has already been successfully used in a similar setting –Initial evaluation of the innovative co-existence approach reveals advantages that had not been foreseen at the time the project was proposed Possibility of draining BoT applications out of the service grid, improving the performance of tightly-coupled applications

Brussels, First Project Review, Work Planned for the 2 nd Year MJRA1.4: Prototype of the gateway to transfer jobs from gLite to OurGrid and vice-versa (M16) MJRA1.5: Stable of the proposed services (M16) MJRA1.6: Stable version of the gateway to transfer jobs from OurGrid to gLite and vice- versa (M22)

Brussels, First Project Review, Plan of Action for the 2 nd Year Development of gateways for grid interoperability and improvement of services developed In addition to the planned developments, promote the dissemination and the use of the services developed –Continue to work in close cooperation with NA3 to identify users that could benefit from the services available and from others that could be developed –Continue to work in close cooperation with NA2 to advertise the services available and to train users to use them –Continue to work in close cooperation with SA1 to refine the prototypes, their documentation and deployment procedures –Continue to participate in grid-related dissemination events –Follow the RESPECT protocol to share the developments Continue research on grid interoperability –Work in synergy with other projects involved in similar efforts, such as the EDGeS project

Brussels, First Project Review, Pointers for more Information JRA1 Resources –EELA-2 web site  –JRA1 Wiki Site  Contains descriptions and links for each JRA1 service  –EELA-Forge Site  All released software  –Contact us at  Francisco Brasileiro