1 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Grid: An LHC user point of vue S. Jézéquel (LAPP-CNRS/Université de Savoie)

2 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Summary First part: Description of the Computing Model Second part: Effective Grid usage ● General remarks ● Experiment usage of the Grid Distributed MC production Distributed analysis

3 S. JEZEQUEL- First chinese-french workshop 13 December 2006 LHC experiments Collision rate : 40 MHz Detectors with 10 6 electronic channels pp or ion/ion collisions : High detector occupancy

4 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Real data production LHCb CMS “30” 2100“1.0” ATLAS ALICE pp ALICE HI Monte Carlo % of real Monte Carlo [MB/evt] AOD [kB] ESD rDST RECO [MB] RAW [MB] Rate [Hz] 10 7 seconds/year pp (2008)  ~2 x 10 9 events/year/experiment  ~5 PB/year/experiment  ~10 TB/day/experiment

5 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Worldwide analysis CMS ATLAS LHCb ~ 5000 Physicists around the world - around the clock “Offline” software effort: 1000 person-years per experiment

6 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Computing Center roles grid for a physics study group Tier3 physics department    Desktop Germany Tier 1 USA UK France Italy Taipei SARA Spain CERN Tier 0 Tier2 Lab a Uni a Lab c Uni n Lab m Lab b Uni b Uni y Uni x grid for a regional group  Collaboration o LHC Experiments o Grid projects: Europe, US o Regional & national centers  Choices o Adopt Grid technology. o Go for a “Tier” hierarchy  Goal o Prepare and deploy the computing environment to help the experiments analyze the data from the LHC detectors.

7 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Computing Model: CMS Tier-1’s:  Real data archiving  Re-processing  Skimming and other data-intensive analysis tasks  Calibration  MC data archiving Tier-2’s:  User data Analysis  MC production  Import skimmed datasets from Tier-1 and export MC data  Calibration/alignment Tier-0:  Accepts data from DAQ  Prompt reconstruction  Data archive and distribution to Tier-1’s

8 S. JEZEQUEL- First chinese-french workshop 13 December 2006 CMS : T1-T2 association

9 S. JEZEQUEL- First chinese-french workshop 13 December 2006 CMS data transfer: Phedex See G. Chen presentation

10 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS : Grid organisation CERN LYON NG BNL FZK RAL CNAF PIC TRIUMF SARA ASGC LPC Tokyo Beijing Romania GRIF T3 SWT2 GLT2 NET2 WT2 MWT2 T1 T2 T3 VO box, dedicated computer to run DDM services T1-T1 and T1-T2 associations according to GP ATLAS Tiers associations. All Tier-1s have predefined (software) channel with CERN and with each other Tier- 1. Tier-2s are associated with one Tier-1 and form the cloud Tier-2s have predefined channel with the parent Tier-1 only. LYON Cloud BNL Cloud TWT2 Melbourne ASGC Cloud “Tier Cloud Model”

11 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS:T1-T2 links

12 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS Computing Model ● Tier-0-1-2: Roles similar to CMS ● Tier-3/Analysis Farm ● Store user data ● Run user analysis/development No particular stream attached to a T1 cloud Each cloud has a complete copy of AOD

13 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS:Data transfers IN2P3 T0 T2 Beijing T0 or T1 (IN2P3) : Mass Storage + Critical Services T1 DDM : ATLAS tool to manage transfers to list files at cloud level DDM Backup Today bottleneck: Transfer-Mass Storage dialog (10 % loss when heavily loaded with painfull recovery)

14 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Challenge: Make all grid components and hardware infracstures work together ATLAS:First tests T0→T1→T2 transfers See G. Rahal presentation End July 2006 No other simultaneous transfer First EGEE operationnal cloud LYON= CC-IN2P3

15 S. JEZEQUEL- First chinese-french workshop 13 December 2006 End-user life with Grid: global remarks

16 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Effective usage of Grid Each day, a new failure of the Grid services occurs Grid implementation (certificates,...) Availability of services at sites See H. Cordier presentation

17 S. JEZEQUEL- First chinese-french workshop 13 December 2006 CMS: Example of failure rate See A. Trunov presentation

18 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Grid: global end-user vision Grid tools completly changed in 2 years and still in development Tool development rely on very few people (→delays) Today situation Submission of jobs on the grid : almost OK Write and read access to data not optimised to be transparent to different storage technologies (Castor, dcache, DPM,...)

19 S. JEZEQUEL- First chinese-french workshop 13 December 2006 First solutions in debugging period Local experts have direct contacts with sites especially T1s, CERN and T2s within cloud (Otherwise use GGUS: response time : Random) Press sites to provide reliable services ATLAS: Recommand users to use few sites at T1 level, close collaboration between IN2P3 (FR), FZK (DE) and BNL (US)

20 S. JEZEQUEL- First chinese-french workshop 13 December 2006 End-user life with Grid: grid usage by ATLAS/CMS

21 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS:Dialog between Athena/Grid ATLAS Software: Athena (No direct interaction with the Grid) Steps: Check possibility of direct access to local data or use Grid commands to transfer them locally Build joboption using informations from the LFC catalog Run Athena job accessing to local files Output files are copied locally or on a grid SE

22 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS:Production of simulated data ● Most reliable tool ● Production occuring in T2/T3 (also T1 now) ● Output data collected in the T1 in the cloud ● AODs are replicated to other T1 ● ESD/AOD can be replicated to T2 sites

23 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS MC Production system EGEE NorduGrid OSG EGEexe EGEE exe NG exe OSG exe super DDM (Data Management) Python DQ2 Eowyn Tasks PanDADulcinea LexorLexor-CG LSF exe super Python T0MS ProdDB (jobs)

24 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS:Monitoring FR cloud: MC prod.

25 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS:Simulation Part of the production chain which is now able to use efficiently the Grid Need one available CPU One input file Few read/write access Submission time << Execution time Needs only few experts to run production tools

26 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS: Production rate Recent evolutions: ● VOMS role (production) attributed to jobs ● Sites requested to implement 80% priority to production jobs Goal end-2006

27 S. JEZEQUEL- First chinese-french workshop 13 December 2006 ATLAS : MC prod. efficiency

28 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Efficiency: CMS

29 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Data Analysis on the Grid

30 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Data analysis:Mid-2006 Mid-2006: Work with simulated data representing few days of data taking Use local copies of data (Replication tool (DDM) starts to be used) Analysis jobs are run locally and interactively or within local batch system

31 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Requirements for Data analysis on Grid Analysis t ools are used by non-grid aware physicists (Should work as prescribed in the twiki) Low level grid commands and limitations due to grid or infrastructure unknown Need simple and reliable tools

32 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Data analysis (2) Key point :Jobs are I/O intensive → Fast access to the data Local access to data better than through Grid commands: ● much faster ● request less ressources on disk servers (more concurrent accesses are possible ) More efficient to send jobs where the data are and use local commands CMS : xroot whenever possible ATLAS: 3 types of commands depending on the SE technology (Castor: rfio Castor / DPM: rfio DPM / dcache : dcap)

33 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Data analysis:End 2006 Analysis outside CERN is starting (with grid) Mandatory: Dataset (collection of files) should be gathered on sites ATLAS: 2 users interfaces on the market: ● Panda (made in BNL) Working efficiently in BNL First trials on LCG ● Ganga (made in Europe: LHCb/ATLAS) Start to be used by european users IN2P3: one of the few validation sites CMS: Crab Working on T1 and T2s

34 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Distributed Data Analysis on T1s Before data taking starts, possible to prepare data analysis on T1 Why start with T1 ? ● where the data are located nowdays ● availability of Grid experts ● huge CPU availability ● already organised to work 24/24 and 7/7 Coordinated work to share data: FZK, BNL and CC-IN2P3 (ATLAS)

35 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Data analysis on T2 One of the key T2 activity ● Beijing T2 stores data on dcache servers also used by CC-IN2P3 → Could benefit directly from the experience in CC-IN2P3 Important to do asap analysis on T2 sites ● to be ready for data taking period ● create common interest between local physicists/site managers

36 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Conclusion Grid tool usage today: become more and more popular not yet robust enough against intensive/chaotic usage ● Simulation of LHC events Now performed only with Grid Next step: improve the efficiency To speed up the production To devote more ressources to analysis Active participation of Beijing T2 is possible

37 S. JEZEQUEL- First chinese-french workshop 13 December 2006 Conclusion (2) Useful data for physicists replicated in T1s and T2s to enable distributed analysis (which data stored in Beijing T2 ?) Distributed analysis in T1s becomes reality Start analysis on T2s Opportunity to start now on Beijing T2 Being efficient with Grid tools will be mandatory to actively participate to LHC analysis groups

38 S. JEZEQUEL- First chinese-french workshop 13 December 2006 With better network connection and more human implication in expt/Grid interface possibility for the Beijing T2 Have a visible participation in LHC grid activities Enable competitive analysis in China in collaboration with some other sites