1. GDB 8th March 2006 Flavia Donno IT/GD, CERN
Software Choices
GDB
8th March 2006
Flavia Donno
IT/GD, CERN

2. Overview Summary of software to be used for SC4 Site Plans
Review of list of services
Experiments requirements
Site Plans

3. Summary of software to be used for SC4 (June 2006)
A reference document has been published:
Such document has been considered during the daily meetings for the definition of the gLite 3.0 release.
It is still quite accurate although package versions have changed due to the bug fixes introduced.
A new updated version of the document will be available tomorrow with the correct package versions.
Still some experiment requirements are left uncovered. However they are not critical.

4. The gLite 3.0 services
VOMS (VOMS API , VOMS admin , VOMS Oracle , VOMS server )
User metadata available
Mirroring available for high availability on-site only. Probably 2-3 VOMS servers deployed.
VOMS groups and roles implemented still by few services (LFC, DPM, WMS)
MyProxy (VDT no VOMS aware)
Automatic proxy renewal for services still not available.
VOMS aware proxy renewal service might be available before June (collaboration with Condor/VDT team)
Automatic proxy renewal with Kerberos token not available.
BDII (v , GLUE )
Work on-going to identify SE type via the GLUE schema. No need to change SE end-point. Durable/Permanent space available as attributes. Need to change the clients that use such info.
No caching of static information for services available

5. The gLite 3.0 services
FTS (agents v , api v /3, fts v , url-copy v )
Automatic retry available through VO agents. Default agent applies the same actions to all conditions.
No automatic error or status notification. Query is always possible. Automatic signalling of error conditions possible via VO plug-ins.
Priority and reshuffling of jobs possible
Plug-ins for catalogues and retries available
No central service for all transfer. Make channel configuration hidden to end-user.
No automatic proxy renewal.
SRM copy and v2 integration on going.
LFC (v )
VOMS enabled.
Some replica attributes available (CMS custodial flag).
DLI interface available
POOL available plug-ins (POOL >= v.2.1.0)
Improved read performance (over LCG 2.6.0; new methods introduced). Load balancing. Bulk operations not available but support for application session and transactions.
Python interface fully operational

6. The gLite 3.0 services Lcg-utils (v.??) GFAL (v 1.7.7-1)
No bulk operations
No integrity check
SRM v2 integration on-going (only v1 available)
Python interface on-going
GFAL (v )
Best replica selection based on location only.
No access to multiple instances of catalogues.
Integration with Rootd on-going (Rootd plug-ins; open/read/write and status operations - basic implementation runs but some developments needed on the GFAL side)
Python interface not available
DPM (v )
SRM v2.1.1 available but not complete (no copy and no global space reservation)
No support for quota, checksum
No Python clients available

7. The gLite 3.0 services RB gLite 3.0 (v.1.5.5-1; client v1.5.9-1)
Interactive access to running jobs information (pick of output sandbox available; no ls or top)
Capability to handle 10**6 jobs/day (improved performance through new Condor-G/C and DAG mechanism. ClassAds recompiled with optimization on).
Improved ranking and matchmaking mechanism
No redundant, fault tollerant service (It depends on redundancy/fault tollerance availability in Condor, Logging and Bookkeeping, cancel and match-making operations)
No possibility of passing job requirements to underlying batch system (memory reqs, etc. - ATLAS)
No caching of sandboxes
No Job latency proportional to expected duration
No support for priorities (user,group,role) at global level
No fair share across users in the same group
CE (gLite 3.0 and LCG 2.7.0)
Two flavors of the CE will be available at the same time (most likely). Only push mode of operation.
No CE interface publicly available (CREAM not available).
CEMON not configured
Combined UI and WNs available. LCG RB will only select LCG CEs.

8. The gLite 3.0 services Glexec (v.0-5-2) G-PBox (v1.0.16)
Not available in gLite 3.0 for applications (available only for CE middleware services). Allows for identity change of jobs on the WNs (LHCb requirement).
G-PBox (v1.0.16)
Not available in gLite 3.0.
R-GMA (v.5.0.n)
VOMS based authentication
No fine-grained VO control of data access
Monitoring of transfer traffic and L&B events
Availability of batch system events (through R-GMA applications)
No monitoring of SE (I/O operations, number of open files, etc.)
GridICE (v )
VO specific information for global operations and jobs available
On-going: sensor for passive monitoring of WMS; filtering of information based on a given BDII; integration of GOC DB downtimes; integration of the L&B job status.
No monitoring of transfer traffic
No monitoring of SE specific information
No monitoring of batch system events
On-going work to monitor L&B events

9. The gLite 3.0 services DGAS (v1.1.n) APEL (v1.0.0) VOBOX (v1.0.1-3)
Installed but not configured in gLite 3.0
Support for granular accounting (user,group,role)
Possibility of aggregation of info per VO
No SE accounting per datasets available
APEL (v1.0.0)
Available in gLite 3.0
Support for granular accounting (EGEE region, user, DN)
No VOMS integration
Aggregation of information per VO (but not per VO tags)
VOBOX (v )
Available in gLite 3.0. Experiments have to negotiate deployment with sites

10. Other experiment requirements
Storage
Common and homogeneous functionality for SRM implementation
A working group is in place. Maarten Litmaath chairing SRM meetings.
Support for quota management
It is still a requirement
Checking for file integrity after file operations.
Availability of optimized SRM client tools with Python binding
On-going
Several SE classes
See proposal from Jeff Templon on using the Glue Schema
Data management
Support for bulk operations
ACL propagation between catalogues and storage
Workload Management System
Support for job prioritization
Information System
Publishing of experiment info (still needed ?)

11. Other experiment requirements
Applications
Resolve conflicts between application and middleware software
A working group is in place. Andrea Sciaba’ is leading the effort and is in contact with the application area.
XRootD
Deployment at all sites
See decision of the Mumbai workshop
Support for Root I/O protocol
Interface to SRM
Batch systems
Support for short jobs
Standards for CPU time limits
Support for queues with at least 2 priority levels
Support for batch systems that can handle priorities at the local queue level

12. Outcome of the pre-CHEP Mumbai Workshop
See final document:
Discussed mainly data management and storage issues
Experiment Production plans
Sites Plans
See Jamie’s presentation

13. Site Plans
The site service proposal and plans available on the Wiki LCG Planning Page:
has been discussed and agreed
During the SC4 weekly meeting a better definition of experiment plans for SC4 and schedule, together with service requirements at the sites is being discussed.
Few issues raised:
Dedicated or shared VO services ?
Network traffic allocation
Resources per VO
Monitoring per VO
…etc.