LHC Computing Grid Project - LCG The LHC Computing Grid First steps towards a Global Computing Facility for Physics 11 November 2003 Les Robertson – LCG Project Leader CERN – European Organization for Nuclear Research Geneva, Switzerland les.robertson@cern.ch

The LHC Accelerator LHC DATA This is reduced by online computers that filter out around 100 “good” events per sec. The LHC accelerator – the largest superconducting installation in the world 27 kilometres of magnets cooled to – 300o C colliding proton beams at an energy of 14 TeV 15 PetaBytes/year The accelerator generates 40 million particle collisions (events) every second at the centre of each of the four experiments’ detectors 15 million DVD movies Which are recorded on disk and magnetic tape at 100-1,000 MegaBytes/sec accumulating a large data base

LHC data analysis Reconstruction: transform signals from the detector to physical properties (energies, charge, tracks, particles, ..) Analysis: Find collisions with similar features, use of complex algorithms to extract physics… Collective iterative discovery of hundreds of physicists – professors and their students Simulation: start from theory and detector characteristics and compute what detector should have seen

LHC data – storage & analysis Balloon (30 Km) LHC data – storage & analysis CD stack with 1 year LHC data! (~ 20 Km) ~15 PetaBytes – about 20 million CDs each year! Its analysis will need the computing power of ~ 100,000 of today's fastest PC processors! Concorde (15 Km) Mt. Blanc (4.8 Km)

Computing for LHC Europe: 267 institutes 4603 users Elsewhere: 208 institutes 1632 users Solution: Computing centres, which were isolated in the past, will be connected into a computing grid CERN has over 6,000 users from more than 450 institutes from around the world -- uniting the computing resources of particle physicists in the world!  

LHC Computing Grid Project The LCG Project is a collaboration of – The LHC experiments The Regional Computing Centres Physics institutes .. working together to prepare and deploy the computing environment that will be used by the experiments to analyse the LHC data This includes support for applications provision of common tools, frameworks, environment, data persistency .. and the development and operation of a computing service exploiting the resources available to LHC experiments in computing centres, physics institutes and universities around the world presenting this as a reliable, coherent environment for the experiments This summarises the key points of the project – -- it is a COLLABORATION of the experiments and the institutes (of which CERN is only one) providing computing resources and manpower for applications development and support -- applications support is a very important part of the project, providing a framework and incentive for common developments -- the project MUST provide a service for analysis of LHC data – using a grid is only a means to this end

LCG Project Applications Area Torre Wenaus Development environment Joint projects, Data management Distributed analysis Middleware Area Frédéric Hemmer Provision of a base set of grid middleware (acquisition, development, integration) Testing, maintenance, support CERN Fabric Area Bernd Panzer Large cluster management Data recording, Cluster technology Networking, Computing service at CERN Grid Deployment Area Ian Bird Establishing and managing the Grid Service - Middleware, certification, security operations, registration, authorisation, accounting Technology Office - David Foster Overall coherence of the project; Pro-active technology watch Long-term grid technology strategy; Computing models

Project Management Board Management Team SC2, GDB chairs Experiment Delegates External Projects EDG, GridPP, INFN Grid, VDT, Trillium Other Resource Suppliers IN2P3, Germany, CERN-IT Architects’ Forum Applications Area Manager Experiment Architects Computing Coordinators Grid Deployment Board Experiment delegates, national regional centre delegates PEB deals directly with the Fabric and Middleware areas

Elements of a Production Grid Service Middleware: - the systems software that interconnects the computing clusters at regional centres to provide the illusion of a single computing facility Information publishing and finding, distributed data catalogue, data management tools, work scheduler, performance monitors, etc. Operations: Grid infrastructure services Registration, accounting, security Regional centre and network operations Grid operations centre(s) – trouble and performance monitoring, problem resolution – 24x7 around the world Support: Middleware and systems support for computing centres Applications integration, production User support – call centres/helpdesk – global coverage; documentation; training

LCG Service Certification and distribution process established Middleware package – components from – European DataGrid (EDG) US (Globus, Condor, PPDG, GriPhyN)  the Virtual Data Toolkit Agreement reached on principles for registration and security Rutherford Lab (UK) to provide the initial Grid Operations Centre FZK (Karlsruhe) to operate the Call Centre The “certified” release was made available to 14 centres on 1 September – Academia Sinica Taipei, BNL, CERN, CNAF, Cyfronet Cracow, FNAL, FZK, IN2P3 Lyon, KFKI Budapest, Moscow State Univ., Prague, PIC Barcelona, RAL, Univ. Tokyo Status on 24 July 2003 – I will email an update on the 11th August There are two activities going on in parallel: -- Pre-production software is being deployed to 10 Regional Centres (Academia Sinica Taipei, BNL, CERN, CNAF, FNAL, FZK, IN2P3 Lyon, Moscow State Univ., RAL, Univ. Tokyo). This is a final validation of the distribution and installation process and tools, and establishes the network of service support people, but the Grid middleware is not the final version yet. About half of the sites have successfully installed this – a few (including BNL and Lyon) are very slow. -- Final testing is under way of the first production set of Grid middleware. This is being done by a group of grid experts from the LHC experiments (the “loose cannons”). There are a few “show stoppers” and about 15 very serious bugs. The target is to install the production software by the end of the month – one week from now!

LCG Service – Next Steps Deployment status – 12 sites active when service opened on 15 September 20 sites now active Pakistan, Sofia, Switzerland, ..preparing to join Preparing now for adding new functionality in November to be ready for 2004 VO management system Integration of mass storage systems Experiments now starting their tests on LCG-1 CMS target is to have 80% of their production on the grid before the end of the PCP of DC04 Essential that experiments use all features (including/especially data management) -- and exercise the grid model even if not needed for short term challenges Capacity will follow readiness of experiments

LCG - Current Active Sites PIC-BARCELONA(pic.ifae.es) IFIC-VALENCIA(ific.uv.es) CIEMAT-MADRID(ciemat.es) UAM-MADRID(ft.uam.es) USC-SANTIAGO-DE-COMPOSTELA(usc.cesga.es) UB-BARCELONA(ecm.ub.es) IFCA-SANTANDER(ifca.unican.es) BNL(bnl.gov) BUDAPEST(kfki.hu) CERN(cern.ch) CSCS-SWITZERLAND(cscs.ch) CNAF(cr.cnaf.infn.it) FNAL(fnal.gov) FZK(fzk.de) KRAKOW(cyf-kr.edu.pl) MOSCOW(sinp.msu.ru) PRAGUE(farm.particle.cz) RAL(gridpp.rl.ac.uk) TAIPEI(grid.sinica.edu.tw) TOKYO(icepp.s.u-tokyo.ac.jp)

Resources in Regional Centres   CPU (kSI2K) Disk TB Support FTE Tape CERN 700 160 10.0 1000 Czech Repub 60 5 2.5 France 420 81 10.2 540 Germany 207 40 9.0 62 Holland 124 3 4.0 12 Italy 507 16.0 100 Japan 220 45 5.0 Poland 86 9 28 Russia 120 30 Taiwan Spain 150 Sweden 179 2.0 Switzerland 26 UK 1656 226 17.3 295 USA 801 176 15.5 1741 Total 5600 1169 120.0 4223 Resources planned for the period of the data challenges in 2004 CERN ~12% of the total capacity Numbers have to be refined – different standards used by different countries Efficiency of use is a major question mark Reliability Efficient scheduling Sharing between Virtual Organisations (user groups) The table of resources gives the numbers that have been provided by the national delegates on the Grid Deployment Board. These resources more or less match the declared needs of the experiments for 2004, after applying some estimates of the efficiency with which we will be able to exploit them in the early days of the grid operation. Resources committed for 1Q04

LCG Service Time-line Level 1 Milestone – Opening of LCG-1 service computing service physics agree spec. of initial service (LCG-1) 2003 2004 2005 2006 2007 first data open LCG-1 (schedule – 1 July) open LCG-1 (achieved) – 1 Sept used for simulated event productions Level 1 Milestone – Opening of LCG-1 service 2 month delay, lower functionality than planned use by experiments will only starting now (planned for end August)  decision on final set of middleware for the 1H04 data challenges will be taken without experience of production running reduced time for integrating and testing the service with experiments’ systems before data challenges start next spring additional functionality will have to be integrated later

LCG Service Time-line computing service physics agree spec. of initial service (LCG-1) 2003 2004 2005 2006 2007 open LCG-1 (achieved) – 1 Sept used for simulated event productions LCG-2 - upgraded middleware, mgt. and ops tools principal service for LHC data challenges Computing model TDRs* LCG-3 – second generation middleware validation of computing models TDR for the Phase 2 grid Phase 2 service acquisition, installation, commissioning experiment setup & preparation Phase 2 service in production first data * TDR – technical design report

LCG and EGEE EU project approved to provide partial funding for operation of a general e-Science grid in Europe, including the supply of suitable middleware – Enabling Grids for e-Science in Europe – EGEE EGEE provides funding for 70 partners, large majority of which have strong HEP ties Similar funding being sought in the US LCG and EGEE work closely together, sharing the management and responsibility for - Middleware – share out the work to implement the recommendations of HEPCAL II and ARDA Infrastructure operation – LCG will be the core from which the EGEE grid develops – ensures compatibility; provides useful funding at many Tier 1, Tier2 and Tier 3 centres Deployment of HEP applications - small amount of funding provided for testing and integration with LHC experiments

Middleware - Next 15 months Work closely with experiments on developing experience with early distributed analysis models using the grid Multi-tier model Data management, localisation, migration Resource matching & scheduling Performance, scalability Evolutionary introduction of new software – rapid testing and integration into mainline services – – while maintaining a stable service for data challenges! Establish a realistic assessment of the grid functionality that we will be able to depend on at LHC startup – a fundamental input for the Computing Model TDRs due at end 2004

Grids - Maturity is some way off Research still needs to be done in all key areas of importance to LHC e.g. data management, resource matching/provisioning, security, etc. Our life would be easier if standards were agreed and solid implementations were available – but they are not We are just entering now in the second phase of development Everyone agrees on the overall direction, based on Web services But these are not simple developments And we still are learning how to best approach many of the problems of a grid There will be multiple and competing implementations – some for sound technical reasons We must try to follow these developments and influence the standardisation activities of the Global Grid Forum (GGF) It has become clear that LCG will have to live in a world of multiple grids – but there is no agreement on how grids should inter-operate Common protocols? Federations of grids inter-connected by gateways? Regional Centres connecting to multiple grids? Running a service in this environment will be challenge!

Human Resources used to October 2003 CERN + applications at other institutes

all staff for LHC services at CERN (inc. networking) Personnel resources 140 FTEs - includes – all staff for LHC services at CERN (inc. networking) Staff at other institutes in applications area Does NOT include EDG middleware without Regional Centres, EDG middleware

Estimated costs at CERN CERN Capacity approx one third of capacity at major centres - This is research – so the “requirements” are constrained to fit the budget expectation