Production Resources & Issues p20.09 MC-data Regeneration MC-Production Status Production Resources & Issues Production Status p20.09 MC-data Regeneration Other Issues Conclusions Tibor Kurča Institut de Physique Nucléaire de Lyon DØ France June 23, 2008 Grenoble June 23, 2008 T.Kurca - D0 France

MC processing resources Grid flavours: 1. Native SAMGrid (SAM+grid tools) 2. Open Science Grid (OSG) 3. LHC computing grid (LCG) - forwarding nodes needed to translate between SAMgrid and OSG/LCG grids Nongrid site CC-IN2P3 - Goal: maximal usage of local & remote resources - Each system has its (dis)advantages June 23, 2008 T.Kurca - D0 France

Prod systems: Pros(Cons) Grid flavours: + access to additional resources - low efficiency (system too complicated) - opportunistic usage  single requests running weeks….. Nongrid site CC-IN2P3 + high efficiency: dependence only on the local resources & separation of production and SAM-related tasks (they are independent) + local storage of the results + local bookkeeping system + flexibility for non-standard requests - manpower intensive - HPSS dependance - LCG dependance June 23, 2008 T.Kurca - D0 France

MC-Production Status Last 12 months 600 M events 50M/month 1.7M/day Sept05-June08 : 1.14 B evts Nice, but …. - goals are much higher: 70M/month, 2.5M/day - too optimistic !!! June 23, 2008 T.Kurca - D0 France

Weekly MC-Production 24M weekly record Exception ! 15M@CC (+d0gstar reprocessing) 17.5M Feb08 short term goal:17.5M/w …. We are far away…. Why ? - Grid efficiency not improving - LCG not operational - CC-worsenning June 23, 2008 T.Kurca - D0 France

MC-Production Rates Optimistic assumptions per week: OSG 4M, SG 4M, CC 5M, LCG 1M max 56M events /month June 23, 2008 T.Kurca - D0 France

ZeroBias Problem p20.09 MC with wrong ZeroBias data overlay - processed with too tight 2.5 Zero-supression  p17 and p20 JES are not the same reconstructed jet energies are lower by 1.5-2.0 % independent of pT Long term solution : regenerate p20.09.02 MC-data with correct ZeroBias running in parallel with new requests… currently done ~186M / 277M events  67% Some good news: all data produced at CCIN2P3 were redone very quickly - all d0gstar files are stored in HPSS (145 TB, 55M events) - about factor 3-4 faster then full simulation chain  done in about 55 days Some bad news : decision to store all p20.09 d0gstar files at FNAL also until new ZeroBias available : …. But it turned out that Grid production is not able to start from d0gstar file June 23, 2008 T.Kurca - D0 France

Other Issues Streamlining the MC-Request System - current procedure is rather time consuming (labour-intensive) & error prone! Goal: reduce manpower intensity of the submission process  keep up with increasing volume of request  build in automatic crosschecks  web page based request system Status: work in progress in collaboration with REX group Access to MC-data - rather painful –> old & new tools (re)appearing June 23, 2008 T.Kurca - D0 France

Acces to MC-Data more tools around - each has some (dis)advantages 2 basic categories of searches - detailed information about the given request ID (Monte Carlo Catalog, request.py script of R. Herber) - list of requests based on the physics constraints like production and/or decay (request.py of R. Herber, script of S. Muanza, CAF-Trees list of F.Couderc) for more details see: http://www-d0.fnal.gov/computing/mcprod/mcc.html  MC-Data   How to find your request IDs Monte Carlo Catalog is being updated for v7 (C.Biscarat, S.Muanza) June 23, 2008 T.Kurca - D0 France

Conclusions Production rate  short term goal 2.5 M events/day (Feb 2008) … But due to persisting/new problems  even 2M/day too optimistic !!! p20.09 MC-data regeneration done at 67% Problems with efficiency of the grid production Production on LCG still down Production @CCIN2P3 slowing down June 23, 2008 T.Kurca - D0 France

Announcements Regeneration of p20.09.02 … from the list by request IDs  tell us if you need higher priority for certain requests MC-production on LCG (private) is possible: thanks to L. Duflot - without SAMGrid interface prerequisites: - production tarballs and ZeroBias data known to LCG (stored at SE) - you belong to the VO enabled on the site Work on streamlining the request system (slow progress) June 23, 2008 T.Kurca - D0 France

MC-Production Status 2 June 23, 2008 T.Kurca - D0 France