Presentation is loading. Please wait.

Presentation is loading. Please wait.

LcgCAF:CDF submission portal to LCG Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel.

Similar presentations


Presentation on theme: "LcgCAF:CDF submission portal to LCG Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel."— Presentation transcript:

1 LcgCAF:CDF submission portal to LCG Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel Jeans, Subir Sarkar and Igor Sfiligoi

2 CDF Italian Computing Group2 CAF CDF Data Handling Model  Fermilab: host data and a farm used mainly to analyze data, the CAF Central Analysis Facility  Remote sites: have a decentralized CAF (dCAF) where Monte Carlo are produced and later copied to FNAL, some sites (ex. CNAF) has data  CAF and dCAF are CDF dedicated farms structured: Head Node Monitor Mailer Submitter User Desktop Worker nodes CafExe MyJob.sh MyJob.exe Developed by: H. Shih-Chieh, E. Lipeles, M. Neubauer, I. Sfiligoi, F. Wuerthwein

3 CDF Italian Computing Group3 Moving to a GRID Computing Model Reasons to move to a GRID computing model  Need to expand resources, luminosity expect to increase by factor 8 until CDF ends  Resource were in dedicated pools, limited expansion and need to be maintained by CDF persons  GRID resources can be fully exploited before LHC experiments will start analyze data  CDF currently access in Europe more than 5000 KSpecInt2000! No hope of having so many resources without GRID

4 CDF Italian Computing Group4 LcgCAF: General Architecture CDF-SE CDF-UI … Robotic tape Storage GRID site CE and worker nodes User Desktop job CDF user must have access to CAF clients and Kerberos V ticket output

5 CDF Italian Computing Group5 Job Submission and Execution CDF-UI Monitor Submitter Mailer Job … WMS WN  Monitor: provide job information to user  Submitter: create JDL. Retrieve user tarball and make it available to WN  Mailer: send mail to user when job is over

6 CDF Italian Computing Group6 Job Monitor CDF-UI Information System Job information WN information Interactive Job Monitor: CafMon kill CafMon tail CafMon dir CafMon log CafMon top CafMon ps WN: job wrapper collects information like load, dir, ps, top WMS : keeps track of job information WMS WN

7 CDF Italian Computing Group7 CDF Code Distribution Lcg site: CNAF Lcg site: Lyon … … Proxy Cache /home/cdfsoft  No CDF code or Run Condition Database need to be available on WN  On demand Parrot translates local warning in HTT requests towards Fermilab code server and DB  To be scalable when a site has many equal requests the information are cached near WN using squid

8 CDF Italian Computing Group8 CDF Output storage  From CDF-SE or CDF storage output copied using gridftp to FNAL fileservers.  An automatic procedure copies files from fileservers to tape and declare them to the catalogue … CDF-SE Job output Worker Nodes … CDF-Storage gridftp KRB5 rcp

9 CDF Italian Computing Group9 Performances SamplesEvents on tape   with 1 Recovery B s ->D s  D s ->  1969964~92%~100% B s ->D s  D s ->K*K2471751~98%~100% B s ->D s  D s ->K s K1774336~92%~100% CDF B Monte Carlo production #  segments succeeded # segments submitted  GRID failures: site miss-configuration, temporary authentication problems, WMS overload LcgCAF failures: Parrot/Squid cache stacked Retry mechanism: necessary to achieve ~100% efficiency WMS retry for GRID failures and LcgCAF “home-made” retry based on logs parsing

10 CDF Italian Computing Group10 Future developments An interface between SAM (CDF catalogue) and GRID services is under development. This will allow:  File declaration into the catalogue directly from LcgCAF  Automatic transfer of files from LCG sites to FNAL tape  Possibility of data analysis on LCG sites by single user

11 CDF Italian Computing Group11 Conclusions SiteCountryKSpecInt2K CNAF-T1Italy1500 INFN-PadovaItaly60 INFN-CataniaItaly248 INFN-BariItaly87 INFN-LegnaroItaly230 IN2P3-CCFrance500 FZK-LCG2Germany2000 IFAESpain20 PICSpain150 UKI-LT2-UCL-HEUK300 LiverpoolUK100  CDF has developed a portal to access the LCG resources  Currently CDF access the following sites:  CDF has produced and is producing Monte Carlo data exploiting these resources demonstrating that GRID can serve also a running experiment


Download ppt "LcgCAF:CDF submission portal to LCG Federica Fanzago for CDF-Italian Computing Group Gabriele Compostella, Francesco Delli Paoli, Donatella Lucchesi, Daniel."

Similar presentations


Ads by Google