Exploiting the Grid to Simulate & Design the LHCb Experiment Eric van Herwijnen (CERN) Glenn Patrick (Rutherford Appleton Laboratory) National e-Science Centre 25th April 2002

Moun Yoke Vertex Shielding Tracker Calorimeters RICH-2 Coil Muon Detector RICH-1 LHCb Experiment Weight = 4,270 tonnes Magnet alone weighs 1500 tonnes! 18m x 12m x 12m 1.2 million electronic channels. Located 100m underground at the Large Hadron Collider (LHC). Due to start operation in 2006.

Large Hadron Collider – CERN, Geneva 7 TeV protons colliding with 7 TeV protons 27km circumference

8000 cryomagnets 8.3 Tesla magnetic field. 40,000 tons of metal cooled to -271°C 1.8 million Amperes without electrical dissipation 40 MWatts needed for cooling LHC Recreating the conditions seconds after the Big Bang 2835 bunches/beam protons/bunch 40MHz crossing rate

Netherlands Brazil France Germany Italy PRC Romania Spain Switzerland Ukraine UK PolandRussia Finland Design and construction of LHCb undertaken by 500 scientists from 48 institutes in 14 countries around the world. Studying the subtle differences between matter and anti-matter. Over 1000 million short-lived particles of matter and anti-matter called B and B-bar mesons will be studied at LHCb each year. In order to design the detector and to understand the physics, many millions of simulated events also have to be produced. b d B meson b d quarks

10 mm Typical decay of B o meson and anti-B o meson. Lifetime is only seconds. Travel a few millimetres before decaying into other particles which have to be detected.

DesignSimulation/Reconstruction Complex interactionsUnderstanding

Grid – A Single Resource Peta Bytes of data storage Many millions of events Many samples Distributed resources Many 1000s of computers required GRID A unified approach Worldwide collaboration Various conditions Heterogeneous operating systems

DataGrid Sites DataGrid Sites Dubna Moscow RAL Lund Lisboa Santander Madrid Valencia Barcelona Paris Berlin Lyon Grenoble Marseille Brno Prague Torino Milano BO-CNAF PD-LNL Pisa Roma Catania ESRIN CERN HEP sites ESA sites IPSL Estec KNMI (>40) Europe Status Testbed 1 Status

Update bookkeeping database Transfer data to mass store Data quality check Submit jobs remotely via Web Execute on farm Analysis

Using Grid Middleware and DataGrid Tools 1. Authentication grid-proxy-init 2. Job submission to DataGrid dg-job-submit 3. Monitoring and control dg-job-status dg-job-cancel dg-job-get-output 4. Data publication and replication globus-url-copy, GDMP 5. Resource scheduling – use of CERN MSS JDL, sandboxes, storage elements

Job Submission dg-job-submit /home/evh/sicb/sicb/bbincl jdl -o /home/evh/logsub/ bbincl jdl: # Executable = "script_prod"; Arguments = " ,v235r4dst,v233r2"; StdOutput = "file output"; StdError = "file err"; InputSandbox = {"/home/evhtbed/scripts/x509up_u149","/home/evhtbed/sicb/mcsend","/hom e/evhtbed/sicb/fsize","/home/evhtbed/sicb/cdispose.class","/home/evhtb ed/v235r4dst.tar.gz","/home/evhtbed/sicb/sicb/bbincl sh","/home /evhtbed/script_prod","/home/evhtbed/sicb/sicb dat","/home/evht bed/sicb/sicb dat","/home/evhtbed/sicb/sicb dat","/home/ evhtbed/v233r2.tar.gz"}; OutputSandbox = {"job txt","D ","file output","file err","j ob txt","job txt"};

Monitoring and Control

Job Local disk Compute Element globus-url-copy Replica Catalogue NIKHEF - Amsterdam CERN TESTBED REST-OF-GRID Job Storage Element replica-get publish register-local-file Storage Element mss Data Data Publishing & Replication

Publish Data on Storage Element Copy data file to storage element: globus-url-copy file:///${chemin}/L69999 gsiftp://lxshare0219.cern.ch/flatfiles/SE1/lhcb/L69999 Register stored data in the catalogue: /opt/globus/bin/globus-job-run lxshare0219.cern.ch /bin/bash -c "export GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf;/opt/edg/bin/gdmp_register_ local_file -d /flatfiles/SE1/lhcb" Publish catalogue: /opt/globus/bin/globus-job-run lxshare0219.cern.ch /bin/bash -c "export GDMP_CONFIG_FILE=/opt/edg/lhcb/etc/gdmp.conf; /opt/edg/bin/gdmp_publish_catalogue -n"

Resource Scheduling: Use of CERN MSS Copy output to MSS: rfcp L /castor/cern.ch/lhcb/mc/L JDL to specify use of CERN MSS: Executable = "script_prod"; Arguments = " ,v235r4dst,v233r2"; StdOutput = "file output"; StdError = "file err"; OutputSE = "lxshare0219.cern.ch"; InputSandbox = {"/home/evhtbed/scripts/x509up_u149","/home/evhtbed/sicb/mcsend", "/home/evhtbed/sicb/fsize","/home/evhtbed/sicb/cdispose.class","/ home/evhtbed/v235r4dst.tar.gz","/home/evhtbed/sicb/sicb/bbincl sh","/home/evhtbed/script_prod","/home/evhtbed/sicb/sicb dat","/home/evhtbed/sicb/sicb dat","/home/evhtbed/sicb /sicb dat","/home/evhtbed/v233r2.tar.gz"}; OutputSandbox = {"job txt","D ","file output","file er r","job txt","job txt"};

Analysis Improved Design