Dirk Duellmann CERN IT/PSS and 3D http://lcg3d.cern.ch LCG 3D Project Update Dirk Duellmann CERN IT/PSS and 3D http://lcg3d.cern.ch

LCG 3D Service Architecture Oracle Streams http cache (SQUID) Cross DB copy & MySQL/SQLight Files M O S S T0 - autonomous reliable service T1- db back bone - all data replicated - reliable service O O F T2 - local db cache -subset data -only local service Online DB autonomous reliable service O M S S R/O Access at Tier 1/2 (at least initially) LCG 3D Status Dirk Duellmann

LCG Database Deployment Plan After October ‘05 workshop a database deployment plan has been presented to LCG GDB and MB http://agenda.cern.ch/fullAgenda.php?ida=a057112 Two production phases March - Oct ‘06 : partial production service Production service (parallel to existing testbed) H/W requirements defined by experiments/projects Based on Oracle 10gR2 Subset of LCG tier 1 sites: ASCC, CERN, BNL, CNAF, GridKA, IN2P3, RAL Oct ‘06- onwards : full production service Adjusted h/w requirements (defined at summer ‘06 workshop) Other tier 1 sites joined in: PIC, NIKHEF, NDG, TRIUMF LCG 3D Status Dirk Duellmann

Oracle Licenses For Tier 1s 3D collected license needs from experiments and s/w projects And validated with T1 site responsibles Some 152 processor licenses (incl. Grid Services and Castor) CERN has negotiated with Oracle a proposal with very attractive conditions T1 sites have been contacted and agreed to the proposal FNAL had already acquired their licenses Tier 1 sites should now be covered for s/w and support! Support accounts for Tier 1s will be enable as soon as papers have been signed and site contribution has been received. LCG 3D Status Dirk Duellmann

Oracle Instant Client Distribution The issue of client distribution has been discussed with Oracle and an agreement has been achieved The instant client can be integrated into the LCG middleware and application distributions As long as the included license file is preserved and adhered to The SPI project in the Application Area will from now on bundle the software as part of AA releases. Experiments and LCG middleware should take advantage and pick up validated client releases from this single source. Version management will happen as for other AA packages via the established channels for external packages LCG 3D Status Dirk Duellmann

Tier 1 Hardware Setup Propose to setup for first 6 month 2/3 dual-cpu database nodes with 2GB or more Setup as RAC cluster (preferably) per experiment ATLAS: 3 nodes with 300GB storage (after mirroring) LHCb: 2 nodes with 100GB storage (after mirroring) Shared storage (eg FibreChannel) proposed to allow for clustering 2-3 dual-cpu Squid nodes with 1GB or more Squid s/w packaged by CMS will be provided by 3D 100GB storage per node Need to clarify service responsibility (DB or admin team?) Target s/w release: Oracle 10gR2 RedHat Enterprise Server to insure Oracle support LCG 3D Status Dirk Duellmann

LCG 3D - Tier 1 Database Setup Phase 1 Sites ASGC, BNL, CNAF, IN2P3, RAL - DB cluster available, part of 3D throughput tests GridKA - DB cluster available, expected to join soon Phase 2 Sites TRIUMF - regular attendance in 3D planning meetings PIC, NIKHEF/SARA - DBA contact established NDGF - early discussions LCG 3D Status Dirk Duellmann

LCG 3D Throughput Tests Initially scheduled for May: Use the production database clusters at tier 1 and obtain a first estimate for the replication throughput which can be achieved with the setup Eg as input to experiment models for calibration data flow Tests started beginning of May, but will extend until end of June Main reasons Server setup problem (often db storage) at sites Firewall configuration Throughput optimization need Oracle experts to be involved LCG 3D Status Dirk Duellmann

June Test Setup Now tests are split into Basic site tests with all T1s LAN throughput optimization with Oracle (CERN local) ATLAS and CMS online groups involved WAN throughput (CERN-CNAF) LFC team and LHCb involved monitoring setup tests (CERN - RAL) Significant load on one central expert at CERN Trying to offload with the help of the database service team LCG 3D Status Dirk Duellmann

Database Monitoring A central Oracle Enterprise Manager repository at CERN has been setup to collect the status and detailed diagnostics of all 3D production clusters. Some sites will in parallel have the information integrated into their site local OEM setups Integration into higher level monitoring tools will be based on this information Eg experiment dashboard, GridView In addition test jobs (a la SFT) are planned as soon as higher level services (eg COOL) are deployed by experiments LCG 3D Status Dirk Duellmann

Preliminary Streams Performance Out of the box replication performance Between 10 and 30 MB/min (LAN) for a single conditions insert job Sufficient for current experiment conditions scenarios Still quite some room for optimisation as neither source nor destination DB seem to be significantly loaded Regular phone calls between Oracle and 3D DBA Head of streams development attending LCG 3D Status Dirk Duellmann

Frontier/SQUID Tests Stress tests with many clients at CERN to validate production setup for CMS DNS based failover validated Setting up an additional node for ATLAS Frontier tests with COOL CMS Frontier tests at FNAL Focus on connection retry and failover Common s/w component shared with database acces CMS July release may pickup LCG AA s/w release which is currently prepared LCG 3D Status Dirk Duellmann

CMS Squid Deployment Squids deployed at several Tier 1 and 2 sites. Tier 1: LCG: ASCC,IN2P3, PIC, RAL, CNAF, FZK, CERN and OSG: FNAL Tier 2:LCG: Bari, CIEMAT, DESY, Taiwan, OSG: UCSD, Purdue, Caltech Remaining Tier 2 sites for SC4 goals: LCG: Belgium, Legnaro, Budapest, Estonia, CSCS, GRIF, Imperial, Rome, Pisa, NCU/NTU, ITEP, SINP, JINR, IHEP, KNU and OSG: Florida, Nebraska, Wisconsin, MIT. In progress, plan to finish in the next month. Request WLCG support for Tier-2 Squid installation Minimum specs: 1GHz CPU, 1GByte mem, GBit network, 100 GB disk. Needs to be well connected (network-wise) to worker nodes, and have access to WAN and LAN if on Private Network. Having 2 machines for failover is a useful option for T-2 but not required. Sites perform Squid installation using existing instructions Installation should be completed and tested by July 15, 2006.

Messages to T1 Sites Initial set of Tier 1s Production Setups need to be usable for throughput tests of the experiments by end of June Need consistent meeting attendance of all sites Need all sites to join the monitoring setup asap so that we can start integration with Need DBA coverage during the experiment tests July-Oct New Tier 1s - NGDF, PIC, NIKHEF/SARA and TRIUMF Production setup needs to be usable for full production in October DB h/w and s/w setup planning will need to start soon (sites should start from current h/w request) Please let us know when your h/w acquisition could/should start LCG 3D Status Dirk Duellmann

Messages to Experiments Need the experiments to confirm or update their current database and/or Frontier requests for October production Need an experiment work plan from each experiment for period from July to October Database and Frontier A significant fraction of the service expected as of October needs to be demonstrated by middle of August The plan should address the full chain from online databases over offline to tier 1 This activity will require significant experiment resources to drive the tests LCG 3D Status Dirk Duellmann

Proposed Timeline End of June Early-July Late-August September October Throughput phase closed, Experiment application and throughput test start Early-July 3D DBA day to plan database setup options with new tier 1 sites. Hosted by one of the new sites? Late-August 3D workshop (experiments and sites) defining October setup and service September Experiment ramp-up test with October Full service open at all tier 1 sites LCG 3D Status Dirk Duellmann