27-28 August 2002Flavia Donno, INFN-Pisa1 Report on GLUE activities Flavia Donno, INFN-Pisa EDG Retreat August 2002

Flavia Donno, INFN-Pisa2 Summary  Resource Discovery and GLUE schema  Authorization services from VO LDAP to VOMS  Common software deployment procedures  IST2002 and SC2002 joint EU-US demos Interoperability : between Grid domains for core grid services, coexistence of optimization/collective services

27-28 August 2002Flavia Donno, INFN-Pisa3 Resource Discovery and GLUE schema Computing Resources Structure Description Entry point into queueing system Container grouping subcluster or nodes Homogeneous collection of nodes Physical computing nodes

27-28 August 2002Flavia Donno, INFN-Pisa4 Resource Discovery and GLUE schema CE Schema implementation status  Logical structure seems to have reached agreement and is stable  Recently added file system, versioning  Interoperability: EDG needs one CE working on a cluster made of a single Subcluster  Information about Hardware are in the subcluster  Information about Queue Status are in the CE  Information providers have to provide Subcluster and CE info  LDAP implementation defined, agreed and released!  Glue Schema will be included in Globus MDS 2.2 (September 15 th )  Globus Info Providers fill the host level info  EDG Info providers filling Subcluster and CE are needed !! When ? Is the 31 st of August confirmed ?  It might be necessary to include the EDG info providers in VDT  This implementation can be tested with MDS 2.1

27-28 August 2002Flavia Donno, INFN-Pisa5 SE Schema implementation status  Logical structure almost agreed  Missing agreement on how to model connection between storage services and file system  LDAP prototype implementation set up  Questions to EDG for release 2: Do we need to model multiple instances of Storage Service on an SE ? Multiple VO etc... Could it be possible to remove the SE concept and introduce the 'storage services' insisting on the same hardware or 'Storage Device‘? Resource Discovery and GLUE schema

27-28 August 2002Flavia Donno, INFN-Pisa6 Resource Discovery and GLUE schema  The US side set up the top-level GIIS mds-vo-name=DataTAG mds-vo-name=iVDGL GIIS structure proposal for the GLUE testbed mds-vo-name=glue

27-28 August 2002Flavia Donno, INFN-Pisa7 GIIS edt004.cnaf.infn.it Mds-vo-name=‘Datatag’ Resource Broker Computing Element-1 PBS WN1 edt001.cnaf.infn.it WN2 edt002.cnaf.infn.it Gatekeeper grid006f.cnaf.infn.it Computing Element -2 Fork/pbs Gatekeeper edt004.cnaf.infn.it GIIS giis.ivdgl.org mds-vo-name=ivdgl-glue dc-user.isi.edu rod.mcs.anl.gov Job manager: Fork hamachi.cs.uchicago.edu iVDGL DataTAG Condor Gatekeeper: US-CMS LSF Gatekeeper: US-ATLAS LSF Gatekeeper: Padova-site GIIS giis.ivdgl.org mds-vo-name=glue Gatekeeper

27-28 August 2002Flavia Donno, INFN-Pisa8 Resource Discovery and GLUE schema GLUE Schema testbed on-going activity Cnaf/Bologna site status - GIIS vo=Datatag edt004.cnaf.infn.it SE v0.1 - EDG , MDS 2.1 Glue Schema CE v1.0, Glue Schema SE v0.1 upgrade to edg1.2 - GRIS grid006f.cnaf.infn.it - EDG , MDS 2.1 Glue Schema CE v1.0 - Upgrade to edg LDAP test (no EDG software, no Globus) gummo.cnaf.infn.it - Glue Schema CE v1.0, Glue Schema SE v0.1 - Configure RB EDG 1.2 in edt003.cnaf.infn.it - Modify the RB to query the Glue Schema - Install EDG/WP4 Information Provider - Install SE EDG 1.2 and Glue Schema SE v0.1

27-28 August 2002Flavia Donno, INFN-Pisa9 Resource Discovery and GLUE schema GLUE Schema testbed on-going activity  US side: Top level GIIS with Glue schema CE v1.0 GRIS with Glue Schema CE v1.0 and MDS 2.1 and static info provider to be upgraded to MDS 2.2 and Globus Info Provider Node with Glue Schema for SE ready for testing.

27-28 August 2002Flavia Donno, INFN-Pisa10 R-GMA implementation and interoperability with MDS  The Glue UML model is general and can have different implementations: LDAP, Relational DB, XML etc.  Schema implementation in R-GMA : implement the Glue UML model in R-GMA and test in the Glue testbed, with EDG WP3 adapt the info providers to fill the the relational table, this should be zero or very low impact Does MDS need to interoperate with R-GMA ? Resource Discovery and GLUE schema

27-28 August 2002Flavia Donno, INFN-Pisa11 Resource Discovery and GLUE schema People and HW resources PEOPLE: DataTAG/WP4.2 people : INFN (1.5 fte) Coordinator:Cristina Vistoli (40%)- INFN CNAF:Sergio Andreozzi (100%)- Vicenzo Ciaschini (20%)- Hardware Resources: Local HW at CNAF/Padova Local HW at ISI iVDGL Contacts: B. Tierney, J. Schopf, E. Deelman, C. Kesselman, A. Shoshani EDG Contacts: M. Sgaravatto, P. Kunstz, J. Gordon, O. Barring

27-28 August 2002Flavia Donno, INFN-Pisa12 Authorization services from VO LDAP to VOMS EDG Tools proposed for use to iVDGL collaborators VO LDAP server VO GroupMAN edg-mkgridmap VOMS under study by iVDGL collaborators. FNAL – Working on KCA/X509 Becoming aware of LCAS/LCMAPS

27-28 August 2002Flavia Donno, INFN-Pisa13 Authorization services People and HW resources PEOPLE: DataTAG/WP4.2 people : INFN (2.5 fte) Coordinator:Roberto Cecchini (30%)- INFN CNAF:Luca Dell’Agnello (30%)- Fabio Spataro (100%)- Vicenzo Ciaschini (80%)- Hardware Resources: Local HW at CNAF iVDGL Contacts: Doug Olson EDG Contacts: Roberto Cecchini

27-28 August 2002Flavia Donno, INFN-Pisa14 Common software deployment procedures Goals:  Contribute to a distribution that: Is EDG and VDT compliant (can be included in VDT/EDG) Includes main EDG and VDT components Allows for interoperation of testbeds Enforces common flexible configuration rulesPlan:  Distribute EDG CA  Distribute EDG Replica Catalogue with GSI  Distribute UI, RB, JSS, L&B as “standalone” components (End of September, edg-release new schema) (End of September, edg-release new schema)  Distribute information providers if requested  Propose a configuration strategy to both VDT and EDG  Work on a common EDG-VDT versioning strategy  Helping experiments with software distribution (CMS/ATLAS PACMAN distribution) (CMS/ATLAS PACMAN distribution) EDG/WP4 Done and installed on GLUE TB Done In progress

27-28 August 2002Flavia Donno, INFN-Pisa15 UI EDG Structure UI RB JSS RB JSS Many light UIsFew heavy RBs SE CE SE CE SE CE Many Ces and SEs LBISRC One LB, IS and RC

27-28 August 2002Flavia Donno, INFN-Pisa16 UI VDT Structure UI RB JSS RB JSS Many VDT Clients SE CE Many VDT Servers LBISRC One LB, IS and RC SE CE SE CE SE CE

27-28 August 2002Flavia Donno, INFN-Pisa17 Common software deployment procedures People and HW resources PEOPLE: DataTAG/WP4.3 people : INFN (1.5 fte) Coordinator:Flavia Donno (20%)- INFN Milano:David Rebatto (30%)- Luca Vaccarossa (50%)- INFN Padova:Sergio Fantinel (50%)- Hardware Resources: Local HW in Milan Local HW in Padova HW in US iVDGL Contacts: Alain Roy, Saul Youseff, Jerry Gieraltowski EDG Contacts: Enrico Ferro

27-28 August 2002Flavia Donno, INFN-Pisa18 Data Movement and Management  It is part of GLUE phase II.  Setting up of working groups in progress with involvement with EDG/WP2.  The existing collaboration between the Globus Team and EDG/WP2 is a good starting point.  Experiment requirements drive the effort. DataTag / iVDGL People: Nikolai Smirnov, David Rebatto, Mike Wilde EDG Contacts: P. Kunstz

27-28 August 2002Flavia Donno, INFN-Pisa19 Joint EU-US grid demos IST November, Copenaghen SC November, Baltimore SC November, BaltimoreGoals: Basic collaboration between European and US grid projects Interoperability between grid domains for applications submitted by users from different virtual organizations Controlled use of shared resources subject to agreed policy Integrated use of heterogeneous resources from iVDGL and EDG testbed domains Infrastructure:  WEB site: Hypernews: Hypernews: Mailing list: Mailing list: Archive: Archive:  GLUE testbed with common schema  VO (DataTAG and iVDGL) LDAP Servers in EU and US  PACMAN cache with software distribution (DataTAG or iVDGL)  Planning document outline

27-28 August 2002Flavia Donno, INFN-Pisa20 Joint EU-US grid demos  GLUE testbed with Common GLUE schema and authorization/authentication tools.  EDG extensions, VDT extensions  “Old” authentication/authorization tools : VO LDAP servers, mkgridmap, etc.  We rely on the availability of the new GLUE schema, RB and Information Providers  Concentrate on visualization: CMS/GENIUS, ATLAS/GRAPPA, EDG/MAPCENTER (EDG/WP7), Farm Monitoring tools (EDG/WP4), iVDGL/GANGLIA, DataTAG/NAGIOS  Use WEB portals for job submission (CMS/Genius, ATLAS/Grappa)  Provide a World Map of the sites involved (EDG-WP7/MapCenter, DataTAG/Nagios)  Monitor job status and statistics (EDG-WP4, DataTAG/Nagios). In Nagios implemented “top users”, “top applications”, “total resources”, “averages over time”, “per VO”,…  Monitor farms (EDG/WP4, DataTAG/Nagios)  Developing plugins/sensors for WP1/L&B info (only on EDG part)  Services Monitoring (EDG/WP4, DataTAG/Nagios [using MDS info])  CMS and ATLAS demo  ATLAS simulation jobs, GRAPPA modified to use both RB or explicit resources  Pythia & CMSIM simulation jobs submitted to intercontinental resources with IMPALA/BOSS interfaced to VDT/MOP, EDG/JDL, Genius portal.  Definition of the application demos in progress.

27-28 August 2002Flavia Donno, INFN-Pisa21 Joint EU-US grid demos

27-28 August 2002Flavia Donno, INFN-Pisa22 Joint EU-US grid demos Joint Task force in Chicago September, 2002 for first DataTAG/iVDGL and ATLAS rehearsal. Work on infrastructure and application demos as much as possible before the middle of September. First simple sumission tests in progress. Hardware definition in progress.

27-28 August 2002Flavia Donno, INFN-Pisa23 Suggestions to EDG for release 2 Work on installation/configuration issues Relocatability Configuration environemnt Service configuration and startup Make sure “vanilla functionality” of core services is kept. Security system with VO support Information System Submission services such as the Globus Gatekeeper, JobManager, Condor-G,... Schedule with time introduction of GLUE components in EDG. Active collaboration. Make service interfaces clear so to allow for easy software sharing and integration whenever possible. Discuss architecture design for Job scheduling, Data Management and Storage Services....