1. Gridifying the LHCb Monte Carlo production system
Eric van Herwijnen, CERN
Tuesday, 19 february 2002
Talk given at GGF4, Toronto

2. Contents LHCb LHCb distributed computing environment
Current GRID involvement
Functionality of current Monte Carlo system
Integration of DataGrid middleware
Monitoring and control
Requirements of DataGrid middleware

3. LHCb LHC collider experiment 109 events * 1Mb = 1 Pb
Problems of data storage, access and computation
Monte Carlo simulation very important for detector design
Need a distributed model
Create, distribute and keep track of data automatically

4. LHCb distributed computing environment
15 countries, 13 European + Brazil, China, 50 institutes
Tier-0: CERN
Tier-1: RAL, IN2P3 (Lyon), INFN (Bologna), Nikhef, CERN + ?
Tier-2: Liverpool, Edinburgh/Glasgow, Switzerland + ? (grow to ~10)
Tier-3: 50 throughout collaboration
Ongoing negotiatons for centres Tier-1/2/3: Germany, Russia, Poland, Spain, Brazil

5. Current GRID involvement
EU DataGrid project (involves HEP, Biology, Medecine and Earth Observation sciences)
Active in WP8 (HEP applications) of DataGrid
Use “middleware” (WP1-5) + Testbed (WP6) + Network (WP7)
Current distributed system works since some time, LHCb is:
Grid enabled, but not Grid dependent

6. MC production facilities (summer 2001)
Centre
Max. (av.) # of CPUs available simultaneously
Batch System
Typical weekly production
% submitted through GRID
CERN
315 (60)
LSF
85 k
10%
RAL
100 (60)
PBS
35k
100%
IN2P3
225 (60)
BQS
Liverpool
300 (250)
Custom
150k 0%
Bologna
20 (20)
Nikhef
40 (40)
Bristol
10 (10)
15k

7. Submit jobs
remotely
viaWeb
Update bookkeeping
database
Transfer data to
Mass store
Execute
on farm
Monitor
performance
of farm via
Web
Data Quality
Check

8. GRID-enabling production
Run mc executable
write log to Web
copy data to mass store
(dg-data-copy)
call CERN servlet
Construct job script and submit via Web
(dg- authentication, dg-job-submit)
mass store
FTP servlet
(dg-data-replication)
copy data to CERN mass store
call servlet to copy data from local mass store to CERN
update bookkeeping db
(?LDAP-now Oracle)

9. Gridi-fying the MC production system
Provide a convenient tool for DataGrid Testbed validation tests
Feed back improvements into the MC system currently in production
Clone current system, replace commands by DataGrid middleware
Report back to WP8 and other workpackages as required

10. Monitoring and control of running jobs
Control system to monitoring distributed production (based on PVSS, author: Clara Gaspar)
Initially for MC production, later all Grid computing
Automatic quality checks on final data samples
Online histograms and comparisons between histograms
Use DataGrid monitoring tools
Feed back improvements into production MC system

12. Requirements on DataGrid middleware
Security: single user logon
Job submission: use “sandboxes” to package environment so that use of AFS is unnecessary
Monitoring: integrate with WP3 tools where possible for farm monitoring, use own tools for data quality monitoring
Data moving: use a single API to move data
We are in a cycle of requirements, design, implementation and testing