1. LHCb Computing Model and Grid Status Glenn Patrick GRIDPP13, Durham – 5 July 2005

2. Glenn PatrickGridPP13 – 5 July 2005 2 LHCb – June 2005 03 June 2005 HCAL MF1-MF3 Mu-filters MF4 LHCb Magnet ECAL

3. Glenn PatrickGridPP13 – 5 July 2005 3 Computing completes TDRs Jan 2000 June 2005

4. Glenn PatrickGridPP13 – 5 July 2005 4 Online System 40 MHz Level-0 Hardware 1 MHz Level-1 Software 40 kHz HLT Software 2 kHz Tier 0 Raw Data: 2kHZ, 50MB/s Tier 1

5. Glenn PatrickGridPP13 – 5 July 2005 5 HLT Output b-exclusivedimuonD*b-inclusiveTotal Trigger Rate (Hz)2006003009002000 Fraction10%30%15%45%100% Events/year (10 9 )263920 200 Hz Hot Stream Will be fully reconstructed on online farm in real time. “Hot stream” (RAW + rDST) written to Tier 0. 2kHz RAW data written to Tier 0 for reconstruction at CERN and Tier 1s. Calibration for proper-time resolution. Clean peak allows PID calibration. Understand bias on other B selections.

6. Glenn PatrickGridPP13 – 5 July 2005 6 Data Flow Reconstruction Brunel Simulation Gauss Digitisation Boole Analysis DaVinci MC Truth Raw Data DST Analysis Objects Stripped DST Framework - Gaudi Detector Description Conditions Database Event Model/Physics Event Model

7. Glenn PatrickGridPP13 – 5 July 2005 7 LHCb Computing Model 14 candidates CERN Tier 1 essential for accessing “hot stream” for 1.First alignment & calibration. 2.First high-level analysis.

8. Glenn PatrickGridPP13 – 5 July 2005 8 Distributed Data RAW DATA 500 TB CERN = Master Copy 2 nd copy distributed over six Tier 1s STRIPPING 140 TB/pass/copy Pass 1: During data taking at CERN and Tier 1s (7 months) Pass 2: After data taking at CERN and Tier 1s (1 month) RECONSTRUCTION 500TB/pass Pass 1: During data taking at CERN and Tier 1s (7 months) Pass 2: During winter shutdown at CERN, Tier 1s and online farm (2months) Pass 3: During shutdown at CERN, Tier 1s and online farm Pass 4: Before next year data taking at CERN and Tier 1s (1 month)

9. Glenn PatrickGridPP13 – 5 July 2005 9 Check File integrity DaVinci stripping Check File integrity DaVinci stripping Check File integrity DaVinci stripping Stripping Job - 2005 Read INPUTDATA and stage them in 1 go Check File status Not yet Staged Prod DB group2 group1 groupN staged Send bad file info Check File integrity DaVinci stripping Good file Merging process DST and ETC ETC DST Send file info Usage of SRM Stripping runs on reduced DSTs (rDST). Pre-selection algorithms categorise events into streams. Events that pass are fully reconstructed and full DSTs written. CERN, CNAF, PIC used so far – sites based on CASTOR.

10. Glenn PatrickGridPP13 – 5 July 2005 10 Resource Profile CPU (MSI2k.yr) 20062007200820092010 CERN0.270.540.901.251.88 Tier-1’s1.332.654.425.558.35 Tier-2’s2.294.597.65 TOTAL3.897.7812.9714.4517.88 DISK (TB) 20062007200820092010 CERN24849682610951363 Tier-1’s7301459243228973363 Tier-2’s71423 TOTAL9841969328140154749 MSS (TB) 20062007200820092010 CERN408825135928574566 Tier-1’s6221244207442857066 TOTAL103020693433714411632

11. Glenn PatrickGridPP13 – 5 July 2005 11 2008 2009 2010 Comparisons - CPU Tier 1 CPU – integrated (Nick Brook) LHCb

12. Glenn PatrickGridPP13 – 5 July 2005 12 Comparisons- Disk LCG TDR – LHCC, 29.6.2005 (Jurgen Knobloch) 54% pledged CERN Tier-1 Tier-2

13. Glenn PatrickGridPP13 – 5 July 2005 13 Comparisons - Tape LCG TDR – LHCC, 29.6.2005 (Jurgen Knobloch) 75% pledged CERN Tier-1

14. 14 DIRAC Architecture DIRAC Job Management Service DIRAC Job Management Service DIRAC CE LCG Resource Broker Resource Broker CE 1 DIRAC Sites Agent CE 2 CE 3 Production manager Production manager GANGA UI User CLI JobMonitorSvc JobAccountingSvc AccountingDB Job monitor InformationSvc FileCatalogSvc MonitoringSvc BookkeepingSvc BK query webpage BK query webpage FileCatalog browser FileCatalog browser User interfaces DIRAC services DIRAC resources DIRAC Storage DiskFile gridftp bbftp rfio Services Oriented Architecture

15. Glenn PatrickGridPP13 – 5 July 2005 15 Data Challenge 2004 DIRAC alone LCG in action 1.8 10 6 /day LCG paused Phase 1 Completed 3-5 10 6 /day LCG restarted 187 M Produced Events 20 DIRAC sites + 43 LCG sites were used. Data written to Tier 1s. Overall, 50% of events produced using LCG. At end, 75% produced by LCG. UK second largest producer (25%) after CERN.

16. Glenn PatrickGridPP13 – 5 July 2005 16 RTTC - 2005 Real Time Trigger Challenge – May/June 2005 150M Minimum bias events to feed online farm and test software trigger chain. Completed in 20 days (169M events) on 65 different sites. 95% produced with LCG sites 5% produced with “native” DIRAC sites Average of 10M events/day. Average of 4,000 cpus Countries Events Produced UK60 M Italy42 M Switzerland23 M France11 M Netherlands10 M Spain8 M Russia3 M Greece2.5 M Canada2 M Germany0.3 M Belgium0.2M Sweden0.2 M Romany, Hungary, Brazil, USA 0.8 M 37%

17. Glenn PatrickGridPP13 – 5 July 2005 17 Looking Forward SC3 LHC Service Operation Full physics run 200520072006 2008 First physics First beams cosmics SC4 Next Challenge SC3 – Sept. 2005 Start DC06 Processing phase May 2006 Alignment/calibration Challenge October 2006 Ready for data taking April 2007 Analysis at Tier 1s Nov. 2005

18. Glenn PatrickGridPP13 – 5 July 2005 18 LHCb and SC3 Phase 1 (Sept. 2005  ): a)Movement of 8TB of digitised data from CERN/Tier 0 to LHCb Tier 1 centres in parallel over a 2 week period (~10k files). Demonstrate automatic tools for data movement and bookkeeeping. b)Removal of replicas (via LFN) from all Tier 1 centres. c)Redistribution of 4TB data from each Tier 1 centre to Tier 0 and other Tier 1 centres over a 2 week period. Demonstrate data can be redistributed in real time to meet stripping demands. d)Moving of stripped DST data (~1TB, 190k files) from CERN to all Tier 1 centres. Phase 1 (Sept. 2005  ): a)Movement of 8TB of digitised data from CERN/Tier 0 to LHCb Tier 1 centres in parallel over a 2 week period (~10k files). Demonstrate automatic tools for data movement and bookkeeeping. b)Removal of replicas (via LFN) from all Tier 1 centres. c)Redistribution of 4TB data from each Tier 1 centre to Tier 0 and other Tier 1 centres over a 2 week period. Demonstrate data can be redistributed in real time to meet stripping demands. d)Moving of stripped DST data (~1TB, 190k files) from CERN to all Tier 1 centres. Phase 2 (Oct. 2005  ): a)MC production in Tier 2 centres with DST data collected in Tier 1 centres in real time followed by stripping in Tier 1 centres (2 months). Data stripped as it becomes available. b)Analysis of stripped data in Tier 1 centres. Phase 2 (Oct. 2005  ): a)MC production in Tier 2 centres with DST data collected in Tier 1 centres in real time followed by stripping in Tier 1 centres (2 months). Data stripped as it becomes available. b)Analysis of stripped data in Tier 1 centres.

19. Glenn PatrickGridPP13 – 5 July 2005 19 SC3 Requirements Tier 1 Read only LCG file catalogue (LFC) for > 1 Tier 1. SRM version 1.1 interface to MSS. GRIDFTP server for MSS. File Transfer Service (FTS) and LFC client/tools. gLite CE. Hosting CE for agents (with managed, backed up file system). Tier 1 Read only LCG file catalogue (LFC) for > 1 Tier 1. SRM version 1.1 interface to MSS. GRIDFTP server for MSS. File Transfer Service (FTS) and LFC client/tools. gLite CE. Hosting CE for agents (with managed, backed up file system). Tier 2 SRM interface to SE. GRIDFTP access FTS and LFC tools. Tier 2 SRM interface to SE. GRIDFTP access FTS and LFC tools. Job Agent WN Job CE Hosting CE LocalSE Software repository Monitor Agent Transfer Agent Request DB

20. Glenn PatrickGridPP13 – 5 July 2005 20 SC3 Resources Phase 1 Temporary MSS access to ~10TB of data at each Tier 1 (with SRM). Permanent access to 1.5TB on disk at each Tier 1 with SRM interface. Phase 1 Temporary MSS access to ~10TB of data at each Tier 1 (with SRM). Permanent access to 1.5TB on disk at each Tier 1 with SRM interface. Phase 2 LCG version 2.5.0 in production for whole of LCG. CPU  MC production: ~250 (2.4GHz) WN over 2 months (non Tier1).  Stripping: ~2 (2.4GHz) WN per Tier 1 for duration of this phase. Storage (permanent)  10TB of storage across all Tier 1s and CERN for MC output.  350 GB disk with SRM interface at each Tier 1 and CERN for stripping output. Phase 2 LCG version 2.5.0 in production for whole of LCG. CPU  MC production: ~250 (2.4GHz) WN over 2 months (non Tier1).  Stripping: ~2 (2.4GHz) WN per Tier 1 for duration of this phase. Storage (permanent)  10TB of storage across all Tier 1s and CERN for MC output.  350 GB disk with SRM interface at each Tier 1 and CERN for stripping output.

21. 21 Data Production on the Grid DIRAC Job Management Service DIRAC Job Management Service PilotJob Agent PilotJob Agent LCG Resource Broker Resource Broker CE 1 CE 2 CE 3 Agent Production DB DIRAC Job Monitoring Service DIRAC Job Monitoring Service Production manager LocalSE RemoteSE File Catalog LFC File Catalog LFC DIRAC CE DIRAC Sites WN Job Agent

22. Glenn PatrickGridPP13 – 5 July 2005 22 UK:Workflow Control Primary event Spill-over event Production Desktop Gennady Kuznetsov (RAL) Gauss BGauss MB Boole BBoole MB Brunel BBrunel MB Sim Digi Reco Software installation Gauss execution Check logfile Dir listing Bookkeeping report Steps Modules

23. Glenn PatrickGridPP13 – 5 July 2005 23 Web Browser Bookkeeping ARDA Server TCP/IP Streaming ARDA Client API Tomcat Servlet ARDA Client API GANGA application UK: LHCb Metadata and ARDA Carmine Cioffi (Oxford)

24. Glenn PatrickGridPP13 – 5 July 2005 24 AtlasPROD DIAL DIRAC LCG2 gLite localhost LSF submit, kill get output update status store & retrieve job definition prepare, configure Ganga4 Job scripts Gaudi Athena AtlasPROD DIAL DIRAC LCG2 gLite localhost LSF + split, merge, monitor, dataset selection UK: Ganga See next talk! Karl Harrison (Cambridge) Alexander Soroko (Oxford) Alvin Tan (Birmingham) Ulrik Egede (Imperial) Andrew Maier (CERN) Kuba Moscicki (CERN)

25. Glenn PatrickGridPP13 – 5 July 2005 25 UK: Conditions Database Data source VersionTime t1t2t3t4t5t6t7t8t9t10t11 VELO alignment HCAL calibration RICH pressure ECAL temperature Production version: VELO: v3 for T t9 HCAL: v1 for T t8 RICH: v1 everywhere ECAL: v1 everywhere Time = T LCG COOL project providing underlying structure for conditions database. User-interface Nicolas Gilardi (Edinburgh)

26. Glenn PatrickGridPP13 – 5 July 2005 26 UK: Analysis with DIRAC Task-Queue Agent Job executes on WN DIRAC Job Installs software Closest SE Data as LFN Matching Check for all SE’s which have data If no data specified Software Installation + Analysis via DIRAC WMS Stuart Patterson (Glasgow) DIRAC API for analysis job submission [ Requirements = other.Site == "DVtest.in2p3.fr"; Arguments = "jobDescription.xml"; JobName = "DaVinci_1"; OutputData = { "/lhcb/test/DaVinci_user/v1r0/LOG/DaVinci_v12r11.alog" }; parameters = [ STEPS = "1"; STEP_1_NAME = "0_0_1" ]; SoftwarePackages = { "DaVinci.v12r11" }; JobType = "user"; Executable = "$LHCBPRODROOT/DIRAC/scripts/jobexec"; StdOutput = "std.out"; Owner = "paterson"; OutputSandbox = { "std.out", "std.err", "DVNtuples.root", "DaVinci_v12r11.alog", "DVHistos.root" }; StdError = "std.err"; ProductionId = "00000000"; InputSandbox = { "lib.tar.gz", "jobDescription.xml", "jobOptions.opts" }; JobId = ID ] PACMAN DIRAC installation tools

27. Half way there! But the climb gets steeper. 2005 Monte-Carlo Production on the Grid 2007 Data Taking Data Stripping Distributed Analysis Distributed Reconstruction Conclusion DC04 DC03