1. INFSO-RI-508833 Enabling Grids for E-sciencE www.eu-egee.org Padova site report Massimo Sgaravatto On behalf of the JRA1 IT-CZ Padova group

2. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 2 CEMon & Information providers How to stay synchronous with LCG ? Last time we agreed to ask LCG to integrate our mods to support the CondorC based CEs, and then rely on the LCG information providers Actually we then found that LCG information providers can support CondorC based CEs as they are (i.e. no changes to the code) –Just a matter of configuration Tested on LSF and PBS CEs –The ones of the prototype CEMon software changed to support these new LCG information providers In the next integration build the support to the LCG information providers will be included In this process of integrating the LCG information providers also contributed in debugging some problems in the LSF dynamic provider

3. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 3 CEMon & R-GMA Publishing of CEMon info also in R-GMA –This can be done via R-GMA Gadget In (Gin)  Not necessary to have a LDAP server (GRIS) –This allows publishing into R-GMA CE (and possibly also CE-SE bind) information –Done and tested on the PBS CE of the prototype (lxb2077) –Instructions (to be used for the CE installer script) sent to the Iteam  No ideas when they will be integrated in the CE installer script CEMon & ServiceTools –This is basically the same service publisher stuff already done to publish the status of the WMS services –Already integrated in the CE installer script –We just needed to fix something in “getInfo” of CEMon

4. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 4 CEMon current activities Working to several enhancements in particular to better support the pull mode, as discussed last time –More powerful “condition” (specifying is the CE is willing or not to receive jobs) specification  More powerful than a simple RegEx  Also discussing with FrancescoP, followed approach to rely on classAds –Setting of expiration in CE info sent by CEMon –In particular, if the “condition” is false, send expired info to client (ISM)  To mark as expired the info in the ISM If the condition is false, it means that the CE doesn’t want to receive new jobs, and so that CE doesn’t have to be matched –Allowing modifying the static subscriptions (the one specified by the CE admin) without the need to restart tomcat

5. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 5 Simplified example of static subscription http://grid005.pd.infn.it:5120/cemonitor/services /CEMonitor 50000 GlueCEStateEstimatedResponseTime<150&&am p;GlueCEStateWaitingJobs<2&&GlueCEInfoLRMStype==&qu ot;lsf" ClassAd SEND_NOTIFICATION TRUE SEND_EXPIRED_NOTIFICATION FALSE

6. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 6 Simplified example of static subscription http://grid005.pd.infn.it:5120/cemonitor/services /CEMonitor 50000 GlueCEStateEstimatedResponseTime<150&&am p;GlueCEStateWaitingJobs<2&&GlueCEInfoLRMStype==&qu ot;lsf" ClassAd SEND_NOTIFICATION TRUE SEND_EXPIRED_NOTIFICATION FALSE Language used to express the “condition”: RegExp or ClassAds

7. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 7 Simplified example of static subscription http://grid005.pd.infn.it:5120/cemonitor/services /CEMonitor 50000 GlueCEStateEstimatedResponseTime<150&&am p;GlueCEStateWaitingJobs<2&&GlueCEInfoLRMStype==&qu ot;lsf" ClassAd SEND_NOTIFICATION TRUE SEND_EXPIRED_NOTIFICATION FALSE RegExp or ClassAds “Condition” expression Must be XML compliant

8. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 8 Simplified example of static subscription http://grid005.pd.infn.it:5120/cemonitor/services /CEMonitor 50000 GlueCEStateEstimatedResponseTime<150&&am p;GlueCEStateWaitingJobs<2&&GlueCEInfoLRMStype==&qu ot;lsf" ClassAd SEND_NOTIFICATION TRUE SEND_EXPIRED_NOTIFICATION FALSE If the condition is true, then sends notifications to client (e.g. ISM)

9. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 9 Simplified example of static subscription http://grid005.pd.infn.it:5120/cemonitor/services /CEMonitor 50000 GlueCEStateEstimatedResponseTime<150&&am p;GlueCEStateWaitingJobs<2&&GlueCEInfoLRMStype==&qu ot;lsf" ClassAd SEND_NOTIFICATION TRUE SEND_EXPIRED_NOTIFICATION FALSE If the condition is false then sends “expired” notifications to client (e.g. ISM)

10. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 10 Migration path First phase –CE ldif info “calculated” by info provider translated into classads –ClassAds evaluation to evaluate if the condition is true or false –Ldif sent to client  No change in the ISM purchaser; just needed to relink with the new API (which supports the new subscription) –Static subscription file can be updated and it is reread on the fly (no necessary to restart tomcat) –Code ready: under test –Not yet committed  Where ? RC1 or HEAD ? Second phase –Classads sent to client instead of ldif –ISM purchaser doesn’t have to perform anymore the ldif  classads translation –Also include expiration of information in classads  Default will be sending expired info when condition is false … but if we are going to work on HEAD and not in RC1, I guess we don’t need to have two distinct phases

11. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 11 CEMon: other issues How can the ISM CEMon purchaser reject notifications from misconfigured or just unwanted CEMons ? CEMon could send notifications on an authenticated channel (using CE credentials), and then AuthZ (via LCAS ?) in the ISM ?

12. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 12 CREAM Code in CVS but not yet linked with the build system (CruiseControl) yet because of a circular dependency Basically submission and cancellation implemented Worked on implementing sandbox staging also via DIME, but some problems found –Decided to postpone this item: with Axis 1.2 (which we hope to rely on after RC1) the scenario will probably be completely different AuthN and AuthZ integrated Missing piece is credential mapping, as we discussed last time –JRA3 contacted: they are planning to provide in the second year a su- exec wrapper  Looks like the right approach, even if all the details are missing  Doesn’t exist yet, and not too clear when it Not detailed plan yet in JRA3 –The other option would be integrating BLAH with LCMAPS Testing and debugging of submit and cancel –Mini testbed in Padova where CREAM and CREAM CLI are tested

13. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 13 CRM workshop Rome, February 17-18 Goals –To enable information exchange about experiences and to discuss about opportunities for common approaches –To make progress towards the definition of standard "compute resource management" (CRM) interface allowing different implementations to co-exist and interoperate  as is happening for the different storage management systems thanks to the SRM interface Attendees –Condor, EGEE/Glite, Globus, INFN-GRID, LCG, NorduGrid, Unicore Outcomes –Agreement to work towards a common Job Description Language  “All the participants of this group are committed to a constructive evaluation of JSDL in the near future” –The area of Compute Resource Interface was considered more complex  Start by comparing the different interfaces  “All the participants of this group will meet again (in Rome in October?) to assess the possibility of developing an agreed set of interfaces for job submit and management” –Agreement to draft a document comparing the approaches to job description and submission of the different projects by 18th April 2005

14. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 14 Prototype and testing activities Maintenance of the WMS of the prototype Support to testing team Weekly prototype meetings Testing of weekly candidate release done according to what was discussed and decided in Catania –This is basically working –What’s the process now that we don’t have to release anymore weekly tags ? Other testing of new stuff Debugging of the CE installer script –Used to install the CE PBS (lxb2077)

15. Enabling Grids for E-sciencE INFSO-RI-508833 Massimo Sgaravatto - INFN Padova 15 Other MPI jobs –Working for Globus based (LCG2) CEs –Basically not tested yet with CondorC-Blah based CEs (because of other priorities)  Likely not be fully working with this week’s tag Work on configuration (JMX) –See report on the configuration meeting Distributed super-scheduling –See Matteo’s talk