1. LHC Computing Grid Project - LCG
The LHC Computing Grid
First steps towards a
Global Computing Facility for Physics
11 November 2003
Les Robertson – LCG Project Leader
CERN – European Organization for Nuclear Research
Geneva, Switzerland

2. The LHC Accelerator LHC DATA
This is reduced by online computers that filter out around 100 “good” events per sec.
The LHC accelerator –
the largest superconducting installation in the world
27 kilometres of magnets cooled to – 300o C
colliding proton beams at an energy of 14 TeV
15 PetaBytes/year
The accelerator generates 40 million particle collisions (events) every second at the centre of each of the four experiments’ detectors
15 million DVD movies
Which are recorded on disk and magnetic tape at 100-1,000 MegaBytes/sec
accumulating a large data base

3. LHC data analysis
Reconstruction: transform signals from the detector to physical properties (energies, charge, tracks, particles, ..)
Analysis: Find collisions with similar features, use of complex algorithms to extract physics… Collective iterative discovery of hundreds of physicists – professors and their students
Simulation: start from theory and detector characteristics and compute what detector should have seen

4. LHC data – storage & analysis
Balloon
(30 Km)
LHC data – storage & analysis
CD stack with
1 year LHC data!
(~ 20 Km)
~15 PetaBytes – about 20 million CDs each year!
Its analysis will need the computing power of ~ 100,000 of today's fastest PC processors!
Concorde
(15 Km)
Mt. Blanc
(4.8 Km)

5. Computing for LHC
Europe:
267 institutes
4603 users
Elsewhere:
208 institutes
1632 users
Solution: Computing centres, which were isolated in the past, will be connected into a computing grid
CERN has over 6,000 users from more than 450 institutes from around the world
-- uniting the computing resources of particle physicists in the world!  

6. LHC Computing Grid Project
The LCG Project is a collaboration of –
The LHC experiments
The Regional Computing Centres
Physics institutes
.. working together to prepare and deploy the computing environment that will be used by the experiments to analyse the LHC data
This includes support for applications
provision of common tools, frameworks, environment, data persistency
.. and the development and operation of a computing service
exploiting the resources available to LHC experiments in computing centres, physics institutes and universities around the world
presenting this as a reliable, coherent environment for the experiments
This summarises the key points of the project –
-- it is a COLLABORATION of the experiments and the institutes (of which CERN is only one) providing computing resources and manpower for applications development and support
-- applications support is a very important part of the project, providing a framework and incentive for common developments
-- the project MUST provide a service for analysis of LHC data – using a grid is only a means to this end

7. LCG Project Applications Area Torre Wenaus Development environment
Joint projects, Data management
Distributed analysis
Middleware Area
Frédéric Hemmer
Provision of a base set of grid middleware
(acquisition, development, integration) Testing, maintenance, support
CERN Fabric Area Bernd Panzer
Large cluster management
Data recording, Cluster technology
Networking, Computing service at CERN
Grid Deployment Area
Ian Bird
Establishing and managing the Grid Service
- Middleware, certification, security
operations, registration, authorisation, accounting
Technology Office - David Foster
Overall coherence of the project; Pro-active technology watch
Long-term grid technology strategy; Computing models

8. Project Management Board
Management Team
SC2, GDB chairs
Experiment Delegates
External Projects
EDG, GridPP, INFN Grid,
VDT, Trillium
Other Resource Suppliers
IN2P3, Germany, CERN-IT
Architects’ Forum
Applications Area Manager
Experiment Architects
Computing Coordinators
Grid Deployment Board
Experiment delegates, national
regional centre delegates
PEB deals directly with the Fabric and Middleware areas

9. Elements of a Production Grid Service
Middleware: - the systems software that interconnects the computing clusters at regional centres to provide the illusion of a single computing facility
Information publishing and finding, distributed data catalogue, data management tools, work scheduler, performance monitors, etc.
Operations:
Grid infrastructure services
Registration, accounting, security
Regional centre and network operations
Grid operations centre(s) – trouble and performance monitoring, problem resolution – 24x7 around the world
Support:
Middleware and systems support for computing centres
Applications integration, production
User support – call centres/helpdesk – global coverage; documentation; training

10. LCG Service Certification and distribution process established
Middleware package – components from –
European DataGrid (EDG)
US (Globus, Condor, PPDG, GriPhyN)  the Virtual Data Toolkit
Agreement reached on principles for registration and security
Rutherford Lab (UK) to provide the initial Grid Operations Centre
FZK (Karlsruhe) to operate the Call Centre
The “certified” release was made available to 14 centres on 1 September – Academia Sinica Taipei, BNL, CERN, CNAF, Cyfronet Cracow, FNAL, FZK, IN2P3 Lyon, KFKI Budapest, Moscow State Univ., Prague, PIC Barcelona, RAL, Univ. Tokyo
Status on 24 July 2003 – I will an update on the 11th August
There are two activities going on in parallel:
-- Pre-production software is being deployed to 10 Regional Centres (Academia Sinica Taipei, BNL, CERN, CNAF, FNAL, FZK, IN2P3 Lyon, Moscow State Univ., RAL, Univ. Tokyo). This is a final validation of the distribution and installation process and tools, and establishes the network of service support people, but the Grid middleware is not the final version yet. About half of the sites have successfully installed this – a few (including BNL and Lyon) are very slow.
-- Final testing is under way of the first production set of Grid middleware. This is being done by a group of grid experts from the LHC experiments (the “loose cannons”). There are a few “show stoppers” and about 15 very serious bugs.
The target is to install the production software by the end of the month – one week from now!

11. LCG Service – Next Steps
Deployment status –
12 sites active when service opened on 15 September
20 sites now active
Pakistan, Sofia, Switzerland, ..preparing to join
Preparing now for adding new functionality in November to be ready for 2004
VO management system
Integration of mass storage systems
Experiments now starting their tests on LCG-1
CMS target is to have 80% of their production on the grid before the end of the PCP of DC04
Essential that experiments use all features (including/especially data management)
-- and exercise the grid model even if not needed for short term challenges
Capacity will follow readiness of experiments

12. LCG - Current Active Sites
PIC-BARCELONA(pic.ifae.es)
IFIC-VALENCIA(ific.uv.es)
CIEMAT-MADRID(ciemat.es)
UAM-MADRID(ft.uam.es)
USC-SANTIAGO-DE-COMPOSTELA(usc.cesga.es)
UB-BARCELONA(ecm.ub.es)
IFCA-SANTANDER(ifca.unican.es)
BNL(bnl.gov)
BUDAPEST(kfki.hu)
CERN(cern.ch)
CSCS-SWITZERLAND(cscs.ch)
CNAF(cr.cnaf.infn.it)
FNAL(fnal.gov)
FZK(fzk.de)
KRAKOW(cyf-kr.edu.pl)
MOSCOW(sinp.msu.ru)
PRAGUE(farm.particle.cz)
RAL(gridpp.rl.ac.uk)
TAIPEI(grid.sinica.edu.tw)
TOKYO(icepp.s.u-tokyo.ac.jp)

13. Resources in Regional Centres
CPU
(kSI2K)
Disk TB
Support
FTE
Tape
CERN
700
160
10.0
1000
Czech Repub 60 5
2.5
France
420 81
10.2
540
Germany
207 40
9.0 62
Holland
124 3
4.0 12
Italy
507
16.0
100
Japan
220 45
5.0
Poland 86 9 28
Russia
120 30
Taiwan
Spain
150
Sweden
179
2.0
Switzerland 26 UK
1656
226
17.3
295
USA
801
176
15.5
1741
Total
5600
1169
120.0
4223
Resources planned for the period of the data challenges in 2004
CERN ~12% of the total capacity
Numbers have to be refined – different standards used by different countries
Efficiency of use is a major question mark
Reliability
Efficient scheduling
Sharing between Virtual Organisations (user groups)
The table of resources gives the numbers that have been provided by the national delegates on the Grid Deployment Board. These resources more or less match the declared needs of the experiments for 2004, after applying some estimates of the efficiency with which we will be able to exploit them in the early days of the grid operation.
Resources committed for 1Q04

14. LCG Service Time-line Level 1 Milestone – Opening of LCG-1 service
computing service
physics
agree spec. of initial service (LCG-1)
2003
2004
2005
2006
2007
first data
open LCG-1 (schedule – 1 July)
open LCG-1 (achieved) – 1 Sept
used for simulated event productions
Level 1 Milestone – Opening of LCG-1 service
2 month delay, lower functionality than planned
use by experiments will only starting now (planned for end August)
 decision on final set of middleware for the 1H04 data challenges will be taken without experience of production running
reduced time for integrating and testing the service with experiments’ systems before data challenges start next spring
additional functionality will have to be integrated later

15. LCG Service Time-line computing service physics
agree spec. of initial service (LCG-1)
2003
2004
2005
2006
2007
open LCG-1 (achieved) – 1 Sept
used for simulated event productions
LCG-2 - upgraded middleware, mgt. and ops tools
principal service for LHC data challenges
Computing model TDRs*
LCG-3 – second generation middleware
validation of computing models
TDR for the Phase 2 grid
Phase 2 service acquisition, installation, commissioning
experiment setup & preparation
Phase 2 service in production
first data
* TDR – technical design report

16. LCG and EGEE
EU project approved to provide partial funding for operation of a general e-Science grid in Europe, including the supply of suitable middleware – Enabling Grids for e-Science in Europe – EGEE EGEE provides funding for 70 partners, large majority of which have strong HEP ties
Similar funding being sought in the US
LCG and EGEE work closely together, sharing the management and responsibility for -
Middleware – share out the work to implement the recommendations of HEPCAL II and ARDA
Infrastructure operation – LCG will be the core from which the EGEE grid develops – ensures compatibility; provides useful funding at many Tier 1, Tier2 and Tier 3 centres
Deployment of HEP applications - small amount of funding provided for testing and integration with LHC experiments

17. Middleware - Next 15 months
Work closely with experiments on developing experience with early distributed analysis models using the grid
Multi-tier model
Data management, localisation, migration
Resource matching & scheduling
Performance, scalability
Evolutionary introduction of new software – rapid testing and integration into mainline services – – while maintaining a stable service for data challenges!
Establish a realistic assessment of the grid functionality that we will be able to depend on at LHC startup – a fundamental input for the Computing Model TDRs due at end 2004

18. Grids - Maturity is some way off
Research still needs to be done in all key areas of importance to LHC
e.g. data management, resource matching/provisioning, security, etc.
Our life would be easier if standards were agreed and solid implementations were available – but they are not
We are just entering now in the second phase of development
Everyone agrees on the overall direction, based on Web services
But these are not simple developments
And we still are learning how to best approach many of the problems of a grid
There will be multiple and competing implementations – some for sound technical reasons
We must try to follow these developments and influence the standardisation activities of the Global Grid Forum (GGF)
It has become clear that LCG will have to live in a world of multiple grids – but there is no agreement on how grids should inter-operate
Common protocols?
Federations of grids inter-connected by gateways?
Regional Centres connecting to multiple grids?
Running a service in this environment will be challenge!

19. Human Resources used to October 2003 CERN + applications at other institutes

20. all staff for LHC services at CERN (inc. networking)
Personnel resources
140 FTEs - includes –
all staff for LHC services at CERN (inc. networking)
Staff at other institutes in applications area
Does NOT include EDG middleware
without Regional Centres, EDG middleware

22. Estimated costs at CERN
CERN Capacity approx one third of capacity at major centres - This is research – so the “requirements” are constrained to fit the budget expectation