1. Slide David Britton, University of Glasgow IET, Oct 09 1 Prof. David Britton GridPP Project leader University of Glasgow GridPP delivering The UK Grid for Particle Physics RCUK e-Science Review 9 th December 2009

2. Slide David Britton, University of Glasgow 2 Monday 23 rd November, 2009 …and in preparation: 200120022003200420052006200720082009 --------- GridPP1 --------- --------- GridPP2 --------- --------- GridPP3 --------- GridPP2+ From Web to GridFrom Prototype to Production From Production to Exploitation The LHC experiments saw first collisions; data was streamed off the detectors; and distributed to a world-wide network of computers. RCUK e-Science Review

3. Slide 3 Institutes CERN computer centre RAL,UK ScotGridNorthGridSouthGridLondon FranceItalyGermanyUSA GlasgowEdinburghDurham Tier 0 Tier 1 National centres Tier 2 Regional groups Offline farm Online system Workstations Useful model for Particle Physics but not necessary for others What is GridPP? David Britton, University of Glasgow GridPP delivers the UK part of the Worldwide LHC Computing Grid (wLCG), a global project that the UK, with funding via GridPP, played a pivotal role in creating. The LHC will produce unprecedented amounts of data, expected to rise beyond 10PB/year. RCUK e-Science Review GridPP is an internationally recognised collaboration of particle physicists and computer scientists from 19 UK universities, STFC and CERN, who have built a Grid for particle physics.

4. Slide National Impact GridPP has successfully developed and deployed a Grid across ~19 sites in the UK with ~20,000 cores and ~11 PB of storage. RAL Tier-1: Oxfordshire. ScotGrid: Durham, Edinburgh, Glasgow. NorthGrid: Lancaster, Liverpool, Manchester, and Sheffield. SouthGrid: Birmingham, Bristol, Cambridge, Oxford, RAL PPD and Jet. London: Brunel, IC, QMUL, RHUL, and UCL. This has changed the landscape of HEP computing in the UK, placing UK physicists at the forefront of research by allowing seamless access to local, national and international resources through the Grid paradigm. David Britton, University of Glasgow RCUK e-Science Review 4 UK Tier-2s – Leverage University funding, Local publicity Tier-2: 15,000 CPUs, 3PB Disk New kit from Viglen at QMUL UK Tier-2s – Enabled development of new University-based facilities New machine room at Glasgow

5. Slide National Impact GridPPs impact has been broader than just the LHC experiments; with support for other experimental collaborations (such as BaBar, D0, Mice, T2K, ILC) and for theoretical physics collaborations (such as UKQCD, and PhenoGrid). The national impact of GridPP goes beyond Particle Physics through the support of a wide range of Virtual Organisations from other disciplines in the UK through the European EGEE project. (Plots show normalised CPU time per VO, Jan- Sep 2009). David Britton, University of Glasgow RCUK e-Science Review 5 All Regions Contribute Supports more than LHC Wide range of VOs

6. Slide National Impact GridPP has complemented the development of the UK National Grid Service, by developing a larger-scale, but more specialised Grid. In so doing GridPP is a foundation stone for the future UK NGI, which will relate to the EU structures. Many GridPP trained staff have moved to external positions, contributing to other areas of science, business, teaching, and software development. David Britton, University of Glasgow RCUK e-Science Review 6 EGI.eu HEP SSC GridPP NGS UK NGI EMI CUE European Framework-7 proposals

7. Slide International Impact David Britton, University of Glasgow RCUK e-Science Review 7 UK funded LCG posts at CERN The UK, with funding via GridPP, played a pivotal role in initiating the Large Hadron Collider Computing Grid (LCG) project by funding 23 posts at CERN for three years. Also via GridPP, the UK played a large role in the EDG and EGEE projects (a set of FP6 and FP7 European projects) that created much of the Grid middleware together with an overlapping European Grid infrastructure that embraces other disciplines in addition to HEP.

8. Slide International Impact David Britton, University of Glasgow RCUK e-Science Review 8 56 Countries 295 Sites 180,000 CPUs 21 Sites 20,000 CPUs Worldwide UK

9. Slide David Britton, University of Glasgow 9 RAL UK Tier-1 – International role Tier-1: 4,500 CPUs, 3PB Disk, 5PB Tape New kit from Viglen at QMUL RCUK e-Science Review International Impact An international role for the RAL Tier-1 as one of 10 worldwide T1 centers. Demanding service level required for receiving and processing data flowing from the LHC. RAL has been central in developing Grid accounting for EGEE and wLCG. RAL hosts the GOCDB, developed by GridPP staff, that collects and records the configuration of all EGEE sites.

10. Slide David Britton, University of Glasgow RCUK e-Science Review 10 International Impact Delivering Resources The UK Tier-1 and Tier-2s provide a large resource contribution to EGEE and wLCG. Delivering Expertise GridPP members play leading roles: RAL started and chaired the European Group of Certification Authorities. RAL leads and chairs the Joint Security Policy Group and Security Vulnerabilities Group. GridPP members have Global responsibilities within the ATLAS and LHCb computing projects. The UK chairs the wLCG Grid Deployment Board and previously the EGEE Project Board. Members have worked on standardization in many areas in the international Open Grid Forum.

11. Slide Wealth Creation David Britton, University of Glasgow RCUK e-Science Review 11 GridPP has supported: imense and iLexIR start-up companies at Cambridge (Camtology – see below) with the GridPP/ATLAS/LHCb developed Ganga interface, and resources. Econophysica (mathamatical models for commodity trading) at QMUL, with access to resources. Total Oil (geoscience research) from Aberdeen, with help establishing a Virtual Organisation to run their applications on the EGEE Grid. GridPPs involvement in wLCG and EGEE has led directly to the formation of Constellation Technologies, a pioneer in Global Grid infrastructure software.

12. Slide Wealth Creation David Britton, University of Glasgow RCUK e-Science Review 12

13. Slide Quality of Life David Britton, University of Glasgow RCUK e-Science Review 13 The Grid allows rapid deployment of large-scale resources to tackle topical problems. In 2006, GridPP resources made the largest contribution to a pan- european search for a cure for bird-flu.

14. Slide David Britton, University of Glasgow RCUK e-Science Review 14 Search for a cure for Malaria: In 2007, the WISDOM docked over 41 million compounds in just six weeks, the equivalent of 80 years work for a single PC. The WISDOM team identified some 5000 interesting compounds, from which they found three families of molecules that could be effective against the malaria parasite. The UK (GridPP) contributed almost half of the total computing resources. Quality of Life

15. Slide Impact on e-Science David Britton, University of Glasgow RCUK e-Science Review 15 GridPPs key impact on e- science is to demonstrate the potential of a Grid for science across multiple disciplines. GridPP has built and runs a Grid in a complex, data- intensive environment at a scale well above anything previously done. A Grid infrastructure now exists that can run tens of millions of jobs annually.

16. Slide Impact on e-Science David Britton, University of Glasgow RCUK e-Science Review 16 In June 2009 a world-wide full-scale test of the computing for LHC data-taking was performed (STEP09): The UK provided over 2 million hours of CPU time in 2 weeks. ATLAS data reprocessing recalled more than 110TB of data from tape (7x higher than nominal rate for 2010 and the best performance world-wide). CMS data was imported to RAL at 100 MB/s and achieved an aggregated recall from tape of 180 MB/s. The UK contributed 32% of the Global CPU time for the LHCb experiment. The Glasgow Tier-2 ran more analysis jobs than any of the other ~60 sites world-wide. STEP09 was a notable success for the UK. Worldwide, the wLCG Grid demonstrated groundbreaking performance. LHCb CPU contributions Jobs at RAL ATLAS reprocessing jobs.

17. Slide Added Value David Britton, University of Glasgow RCUK e-Science Review 17 The GridPP project added value by: Creating and running a Grid that allows resources to be shared locally, nationally, and internationally. By supporting the development of Ganga, the RealTime Monitor, GridSite, R-GMA and many other utilities to some degree. Supporting up to 40 Virtual Organisations across many disciplines; allowing the opportunistic use of available resources and an opportunity to develop experience with the Grid environment. Enabling commercial interests to obtain experience with the Grid and gain the use of significant computational resources to develop products. By driving technology, such as light-path networking and low-cost, high-performance computational resources. By developing innovative management tools that were used by other projects. By operating in a cutting-edge, complex environment that attracted funding for a study by the Pegasus project at the London School of Economics.

18. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 18 Please provide any available data, or at least anecdotes, about how the e-science program to date has in fact enhanced collaboration in you area of research. The LHC experiments require collaboration on a global scale to tackle the volume and complexity of the data from the LHC. The worldwide LHC Computing Grid, of which GridPP represents 10-15%, is an essential infrastructure, without which the LHC data could not be analysed. In essence, the e-infrastructure has realized the goal of global collaboration in High Energy Physics at a scale that could not be supported by the traditional computer centre. Enhanced collaboration has been observed: within the regional groups of universities making up the distributed Tier-2s, driven by GridPP organisation and funding; between the Tier-2s and with the UK Tier-1, driven by the GridPP deployment team and support structure and by sharing the UK contribution to WLCG and EGEE. between the UK and other countries both within the WLCG and EGEE projects. The coordination of GridPP has allowed the UK to present a credible, organized and common face to these international projects

19. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 19 What evidence (hard data or anecdotes) of data sharing has occurred in your e- science project? The STEP09 exercise in May 2009 exercised the worldwide LHC Computing Grid at full scale and across all areas. Data were shipped out from CERN via the OPN optical network to the ten Tier-1 centers across the world. There, it was reconstructed, archived, restored, and derived data sets made available to the network of Tier-2 centers. At the latter, the data were analysed and physics quantities extracted. This exercise demonstrated that the infrastructure works at, and in many areas, beyond, the levels required for data taking in 2010 and demonstrates both the divide-and-conquer philosophy of the Grid, and the intrinsic sharing of physics data across the globe.

20. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 20 What evidence of sharing mechanisms or services has occurred in your e-science project? Through e-science funding of GridPP, the UK has been a major contributor to both wLCC and EGEE. In the year to July 2009, GridPP provided 17% of EGEEs computer time and contributed nearly 85 million CPU hours (KSI2K hours) to more than 40 EGEE virtual organizations across a wide range of disciplines. Of this, approaching 70 million CPU hours were provided to LHC VOs, 14% of WLCGs worldwide total. The entire wLCG infrastructure is a mechanism and service for sharing data between members of a recognized collaboration. Collaborations are recognized via the Virtual Organisation Management System (VOMS) that verifies the membership and rights of individual. Higher-level tools are provided to help locate and move data (the LHC file catalogue, LFC, and the File Transfer Service, FTS). GridPPs production grid includes a variety of services that are shared by all UK institutes and users, such as the BDII Information Service; Workload Management Services and Nagios monitoring.

21. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 21 What software have you built that has been used outside your project? GANGA – User interface and job submitter – used extensively by HEP, other disciplines and commercial users. Described in later talks. APEL accounting – used to populate the GOCDB with information on all sites in EGEE. GridSite - a certificate-based website management system, that allows members of an organisation to collaborate in maintaining web pages etc. Used to create a dozen or so external (to HEP) websites. The RealTimeMonitor – high profile tool produced by GridPP; used throughout the world to demonstrate the Grid in action. R-GMA – an information and monitoring tool developed at RAL, which was been deployed in LCG middleware and gLite. Contributions to many others.

22. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 22 What disciplines have been involved in your project? Describe their interaction. GridPP is fundamentally a collaboration between Particle Physicists and Computer Scientists. The PEGASUS project based at LSE studied GridPP over a number of years – involved a lot of interaction (observation; interviews; talks). GridPP supports over 40 Virtual Organisations covering a wide range of scientific disciplines.

23. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 23 What major investments made by the e-science core program and/or JISC (such as the National Grid Service) have you used in your research? GridPP is dependent on the academic network hosted by JANET-UK and the dedicated OPN optical network to CERN. GridPP developed the original UK Certificate Authority and then passed this over to the NGS/JISC to run.

24. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 24 What has your project contributed back to the larger research community? Leadership roles (in ATLAS, LHCb, wLCG, EGEE, international security) Software (already described). The UK CA just mentioned. Trained staff who have moved on into the wider world. Standardisation work in the international OGF context (eg information schema). Resources for 40 VOs. Driven technology – pushed agenda for lightpath service in the UK (UKLIGHT project).

25. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 25 What has been the major impact of the e-science program to date on your research and allied education program? The e-science programme has enabled the UK to participate in the worldwide e- infrastructure for LHC computing: without the e-Science program, UK physicists would not be in a position to exploit the data expected shortly from the LHC.

26. Slide Panel Questions David Britton, University of Glasgow RCUK e-Science Review 26 What is your vision for e-science going into the future? Stress what is most critical to the future of your and/or your communities research aspirations? See next slide.

27. Slide Conclusion David Britton, University of Glasgow 27 GridPP has achieved everything that it set out to do, but this is the start, not the end point: The Grid will have to serve the UK particle physics community and others, for the next 15-20 years. It will need to evolve to deal with increasing demands (e.g. SuperLHC luminosities), cope with new discoveries (e.g. Higgs), and respond to new challenges (e.g. ILC). It will need to incorporate new technology (e.g. many-core processors, GPUs), adapt to new structures (e.g. the UK NGI), and service a growing number of communities without losing focus. What is needed is the sort of forward-thinking vision of Sir John Taylor in 2001 accompanied by a plan for long-term support. RCUK e-Science Review