1. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting GridPP Status Report Tony Doyle

2. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Contents What was GridPP1? What is GridPP2? Challenges abound LCG –Issues Deployment Status (9-28-30/1/05 ) –UK Grid The UK mountain climb Summary

3. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting What was GridPP1? A team that built a working prototype grid of significant scale > 2,000 (9,000) CPUs > 1,000 (5,000) TB of available storage > 1,000 (6,000) simultaneous jobs A complex project where 88% of the milestones were completed and all metrics were within specification A Success The achievement of something desired, planned, or attempted

4. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Executive Summary I The GridPP1 Project is now complete: following 3 years of development, a prototype Grid has been established, meeting the requirements of the experiments and fully integrated with LCG, currently the Worlds largest Grid. Starting from this strong foundation, a more complex project, GridPP2, has now started, with an extended team in the UK working towards a production Grid deployed for the benefit of all experiments by September 2007. We achieved (almost exactly) what we stated we would do in building a prototype…

5. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Executive Summary II 2004 was a pivotal year, marked by extraordinary and rapid change with respect to Grid deployment, in terms of scale and throughput. The scale of the Grid in the UK is more than 2000 CPUs and 1PB of disk storage (from a total of 9,000 CPUs and over 5PB internationally), providing a significant fraction of the total resources required by 2007. A peak load of almost 6,000 simultaneous jobs in August, with individual Resource Brokers able to handle up to 1,000 simultaneous jobs, gives confidence that the system should be able to scale up to the required 100,000 CPUs by 2007. A careful choice of sites leads to acceptable (>90%) throughput for the experiments, but the inherent complexity of the system is apparent and many operational improvements are required to establish and maintain a production Grid of the required scale. Numerous issues have been identified that are now being addressed as part of GridPP2 planning in order to establish the required resource for particle physics computing in the UK. Most projects fail in going from prototype to production… There are many issues: methodical approach reqd.

6. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting What is GridPP2? Structures agreed and in place (except LCG phase-2) 253 Milestones, 112 Monitoring Metrics at present. Must deliver a Production Grid: robust, reliable, resilient, secure, stable service delivered to end-user applications. The Collaboration aims to develop, deploy and operate a very large Production Grid in the UK for use by the worldwide particle physics community.

7. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Must share data between thousands of scientists with multiple interests link major (Tier-0 [Tier-1]) and minor (Tier-1 [Tier-2]) computer centres ensure all data accessible anywhere, anytime grow rapidly, yet remain reliable for more than a decade cope with different management policies of different centres ensure data security be up and running routinely by 2007 What are the Grid challenges?

8. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting What are the Grid challenges? Data Management, Security and Sharing 1. Software process 2. Software efficiency 3. Deployment planning 4. Link centres 5. Share data 6. Manage data7. Install software 8. Analyse data9. Accounting 10. Policies

9. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Where do we start? Issues https://edms.cern.ch/file/495809/2.2/LCG2-Limitations_and_Requirements.pdf First large-scale Grid production problems being addressed… at all levels LCG-2 MIDDLEWARE PROBLEMS AND REQUIREMENTS FOR LHC EXPERIMENT DATA CHALLENGES Overall efficiency ~60% -> ~90% ¼ -> ½ of the problems ¾ -> ½ of the problems

10. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting GridPP Deployment Status (9-28-30/1/05) Three Grids on Global scale in HEP (similar functionality) sitesCPUs LCG (GridPP)104 (16) 10,000 (2242) Grid3 [USA]292800 NorduGrid303200 GridPP deployment is part of LCG (Currently the largest Grid in the world) The future Grid in the UK is dependent upon LCG releases totalCPUfreeCPUrunJobwaitJobseAvail TBseUsed TBmaxCPUavgCPU Total2242915591784936.874.45106482232

11. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting UK Grid The whole is better than the sum of the parts..

12. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Applications There is a (slightly wonky?) wheel Use it to get to where you need to be ZEUS uses LCG needs the Grid to respond to increasing demand for MC production up to 6 million Geant events per week on Grid since August 2004 1.The system developed for the large LHC experiments works (more) effectively for other (less resource- intensive) applications 2.Experiments need to work together with deployment team/sites 3.The de-facto deployment standard is LCG – it ~works. We can add components as required, to meet each experiments needs

13. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Dissemination much has happened.. more people are reading about it.. GridPP2 gets its first term reportFri 28 Jan 2005 BaBar UK moves into the Grid eraTue 11 Jan 2005 LHCb-UK members get up to speed with the GridWed 5 Jan 2005 GridPP in PittsburghThu 9 Dec 2004 GridPP website busier than everMon 6 Dec 2004 Optorsim 2.0 releasedWed 24 Nov 2004 ZEUS produces 5 million Grid eventsMon 15 Nov 2004 CERN 50th anniversary receptionTue 26 Oct 2004 GridPP at CHEP'04Mon 18 Oct 2004 LHCb data challenge first phase a success for LCG and UKMon 4 Oct 2004 Networking in Nottingham - GLIF launch meetingMon 4 Oct 2004 GridPP going for Gold - website award at AHMMon 6 Sep 2004 GridPP at the All Hands MeetingWed 1 Sep 2004 R-GMA included in latest LCG releaseWed 18 Aug 2004 LCG2 administrators learn tips and tricks in OxfordTue 27 Jul 2004 Take me to your (project) leaderFri 2 Jul 2004 ScotGrid's 2nd birthday: ScotGrid clocks up 1 million CPU hoursFri 25 Jun 2004 Meet your production managerFri 18 Jun 2004 GridPP10 report and photographsWed 9 Jun 2004 CERN recognizes UK's outstanding contribution to Grid computingWed 2 Jun 2004 UK particle physics Grid takes shapeWed 19 May 2004 A new monitoring map for GridPPMon 10 May 2004 Press reaction to EGEE launchTue 4 May 2004 GridPP at the EGEE launch conferenceTue 27 Apr 2004 LCG2 releasedThu 8 Apr 2004 University of Warwick joins GridPPThu 8 Apr 2004 Grid computing steps up a gear: the start of EGEEThu 1 Apr 2004 EDG gets glowing final reviewMon 22 Mar 2004 Grids and Web Services meeting, 23 April, LondonTue 16 Mar 2004 EU DataGrid Software License approved by OSIFri 27 Feb 2004 GridPP Middleware workshop, March 4-5 2004, UCLFri 20 Feb 2004 Version 1.0 of the Optorsim grid simulation tool released by EU DataGridTue 17 Feb 2004 Summary and photographs of the 9th GridPP Collaboration MeetingThu 12 Feb 2004 138,976 hits in December

14. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Annual data storage: 2.4-2.8 PetaBytes per year? (~20%) 10 Million SPECint2000 10,000 PCs (3 GHz Pentium 4) CD stack (~ 4 km) The UK mountain climb has started.. Quantitatively, were ~10% of the way there in terms of UK CPU (~2,000 ex ~10,000) and disk (~1 ex ~10 PB) In production terms, left base camp We are here (0.4 km) step-by-step plan in place… For the Ben Nevis climb? totalCPUfreeCPUrunJobwaitJobseAvail TBseUsed TBmaxCPUavgCPU Total2242915591784936.874.45106482232

15. Tony Doyle - University of Glasgow 3 February 2005Science Committee Meeting Summary GRIDPP-PMB-40-EXEC The Grid is a reality A project was/is needed Under control LCG2 support: SC case presn. 3/2/05 16 UK sites are on the Grid –MoUs, planning, deployment, monitoring –each underway as part of GridPP2 Developments estd., R-GMA deployed gLite designed inc. web services Interfaces developed, testing phase Area transformed Incorporation in HEP programme.. Introduction Project Management Resources LCG Deployment –Tier-1/A production + Tier-2 resources M/S/N EGEE Applications Dissemination Beyond GridPP2