1. Tony Doyle a.doyle@physics.gla.ac.uk GridPP2 – Project Specification, GridPP9 Collaboration Meeting, Edinburgh, 4 February 2004

2. Tony Doyle - University of GlasgowOutline Background… A Year in the Life ARDA first thoughts Patchwork Quilting PMB High Level View EGEE Grid Operations GridPP2 Deployment Board GridPP2 User Board Who reports where? ARDA second thoughts We are legion.. Reporting: General Points Proposed Structure Where does the work get done? Areas on Green Alert Areas on Amber Alert Outcome?

3. Tony Doyle - University of Glasgow 4. Background… GridPP2 steps towards project spec. 0. PPARC Call (February 2003) 1.GridPP2 Proposal (30/5/03) 2. Feedback from Reviewers 3. Responses from the PMB See http://www.gridpp.ac.uk/docs/gridpp2/ 4. Projects Peer Review Panel (14-15/7/03) 5. Grid Steering Committee (28-29/7/03) See http://www.gridpp.ac.uk/talks/GridPP_GSC_030728.ppt 6. Tender Document (11/11/03) See http://www.gridpp.ac.uk/cb/doc/GridPP2_Tender.doc 7. Science Committee (17-18/11/03) 8. PPARC (25/11/03) 9. CB (4/12/03) 10.PRSC (16/1/04). PRSC summary 11.PMB (26/1/04). Work area effort tables 12. GridPP9 Collaboration Meeting (4-5/2/04) 13.OC (18/2/04)… GridPP2 presentation. (unlucky for some..)

4. Tony Doyle - University of Glasgow 3. Background… A Year in the Life of Globus

5. Tony Doyle - University of Glasgow 2. Background… UK e-Science Development 1.The UK e-Science Production Grid will have significant overlap with the GridPP2 Production Grid –Mutual benefit to engagement –e.g. convergence talks organised prior to Christmas –GT2-based approach 2.OMII developments will be based upon the Web Services Resource Framework –The OMII delivery timescales are in 2005 onwards –Incorporates e.g. OGSA-DAI –Limited overlap with our proposed programme

6. Tony Doyle - University of Glasgow 1. Background… LCG Development 1.AliEn provided a pre-Grid implementation 2.ARDA provides a framework for PP application middleware –We can and should (and do) participate here –Identify your development within this structure

7. Tony Doyle - University of Glasgow ARDA thoughts.. Ian Stokes-Rees ARDA is the LCG middleware – framework to deliver a distributed analysis environment to the experiments. We need to look at the decomposition and contribute coherently to this There are existing products which need to be used as building blocks here, e.g. –GANGA=Grid Job Submission Wizard –DIRAC=Distributed MC Production Service –Spitfire = Metadata Service –R-GMA = Information Service http://www-pnp.physics.ox.ac.uk/~stokes/arda/ARDA-views-ijstokes-20040122.pdf

8. Tony Doyle - University of Glasgow Patchwork Quilting the GridPP2 HourGlass Tender DocumentOutcome Tier-1 Applications Middle/Sec/Net Tier-2 + 1 Portal + 1 PhenoGrid -0.5 BaBar -1 LHC Deployment +1 LHC Deployment-1 Network -0.5 Workload -1 Data Management +1 Data Management III. Grid Middleware e.g. Information Services I. Experiment Layer e.g. Portals II. Application Middleware e.g. Metadata IV. Facilities and Fabrics e.g. Storage Services

9. Tony Doyle - University of Glasgow SysManExperiments Developers Deployment Board User Board Dissemination Team Developer Team GridPP2 PMB High Level View Opinion Formers PMB

10. Tony Doyle - University of Glasgow PMB User Board Developer Team GridPP2 Organisational View? (~75 FTEs) Deployment Board Dissemination Team

11. Tony Doyle - University of Glasgow Grid and Regional Operations Summary Equation: GOC > ROC (= ROC+CIC) –UK has a leading role.. –Grid Operations Centre a.k.a. "International Rescue –But if its that exciting by the end of GridPP2, then we will have failed.. Regional Operations Centre –Core Operational Tasks –Monitor infrastructure, components and services –Troubleshooting –Verification of new sites joining Grid –Acceptance tests of new middleware releases –Verify suppliers are meeting SLA –Performance tuning and optimisation –Publishing use figures and accounts –Grid information services –Monitoring services –Resource brokering –Allocation and scheduling services –Replica data catalogues –Authorisation services –Accounting services

12. Tony Doyle - University of Glasgow EGEE Grid Operations EGEE GridPPJISCCore UK +1 + Technical Editor ~ 23 FTE (EU + non-EU funded) The characters are the same whichever OS

13. Tony Doyle - University of Glasgow SysMan GridPP2 Deployment Board Foundation Viewpoint Production Manager Grid Support Centre 5 FTE Core Grid Coordinator 1 FTE MiddleWare Specialist Support 6 FTE Deployment Team 8 FTE Grid Operations Centre 3 FTE Summary: The Production Manager & DB have a major role in (re-)defining and implementing Production Planning. Refs: GridPP-PMB-24-Production GridPP-PMB- 20-Management III. Grid Middleware e.g. Information Services IV. Facilities and Fabrics e.g. Storage Services Deployment Board

14. Tony Doyle - University of Glasgow SysMan GridPP2 Deployment Board: Two Roles The role of Production Manager is to: 1.lead the Production Grid Team; 2.sit on the Deployment Board; 3.ensure the formulation and implementation of procedures necessary to ensure timely deployment; 4.report to the Deployment Board on progress and issues; 5.liaise with all the Tier centres on the technical implementation of the production grid. The role of the Deployment Board Chair is to: 1.convene and chair Deployment Board meetings; 2.sit on the Project Management Board; 3.ensure that the Deployment Board are kept fully aware of the objectives of the project and the priorities identified by the PMB; 4.ensure that the Project Management Board is full aware of technical progress and any issues that arise; 5.ensure that progress toward deliverables is monitored against the project schedule on a quarterly basis; 6.ensure that the use of GridPP resources is monitored and reported; 7.report to the Project Manager on matters relating to any project deliverables and milestones. Deployment Board

15. Tony Doyle - University of Glasgow SysMan GridPP2 Deployment Board Terms of Reference The Deployment Board is the body at which technical developments and problems wider than a single Working Group or Tier Centre are discussed and solved. The Deployment Board meets approximately quarterly to: 1.determine and execute the technical project plan necessary to implement the strategy of the Project Management Board on the deployment of a production quality grid for Particle Physics; 2.report to the Project Management Board, through its chair, on progress towards this goal; 3.ensure that problems and ideas with implications beyond individual working groups are discussed and solved in the most efficient manner; 4.provide a forum in which all constituent parts of the project can exchange information on technical issues of mutual interest; 5.manage the deployment of GridPP2 hardware resources in the UK, ensuring they are efficiently and effectively used to deliver the Particle Physics Grid with the minimum of duplication; 6.deal with matters of coordination and liaison between UK Tier1 and Tier2 centres; 7.ensure that technical developments elsewhere, particularly at CERN, are efficiently integrated into the project. Deployment Board

16. Tony Doyle - University of Glasgow User Board Experiments The User Board ensures that the requirements for the major Particle Physics experiments that will use the Grid are properly taken into account in the strategy of the PMB. The User Board meets approximately quarterly to: 1.ensure that the GridPP2 project remains science driven and that the project develops to meet the needs of the UK Particle Physics community; 2.keep an overview of both technical and experimental software developments, ensuring these developments are compatible; 3.facilitate close working relationships between the UK experimental collaborations to identify common areas for the development of Grid-enabled experimental software; 4.overview the major hardware purchases within GridPP2 to ensure compatibility with the needs of the experimental collaborations and between collaborations; 5.promote the use of the developing Grid structures for a wide variety of computing needs in the whole of the UK Particle Physics community. GridPP2 User Board Terms of Reference I. Experiment Layer e.g. Portals II. Application Middleware e.g. Metadata

17. Tony Doyle - University of Glasgow The PMB is the central part of the management structure. The Project Management Board meets approximately weekly to: 1.determine the detailed objectives of the project; 2.approve the allocation of resources to meet these objectives; 3.ensure timely progress is made towards meeting the objectives; 4.report on progress to the Oversight Committee; 5.inform the community of progress, formally via the Collaboration Board; 6.establish an identity for GridPP2 reflecting the set of GridPP deliverables; 7.ensure that GridPP2 is visible externally and is recognised as a flagship project. GridPP2 Project Management Board Terms of Reference III. Grid Middleware e.g. Information Services I. Experiment Layer e.g. Portals II. Application Middleware e.g. Metadata IV. Facilities and Fabrics e.g. Storage Services PMB User Board Deployment Board PMB

18. Tony Doyle - University of Glasgow Who reports where? Developers Developer Team Dissemination Officer Opinion Formers Representatives Formal roles are (pre-)defined in the proposal GridPP-PMB-20- Management

19. Tony Doyle - University of Glasgow GAG LCG PEB AliceATLASCMSLHCb Generic Middleware Resource providers Requirements Guidelines Service specs Integration: workers forum ARDA – A Realisation of Distributed Analysis LCG AA projects

20. Tony Doyle - University of Glasgow ARDA: many facets

21. Tony Doyle - University of Glasgow Application Interfaces [FTE] Applications –18.5 FTEs: ongoing programme of work can continue –Difficult to involve experiment activity not already engaged within GridPP Project needs to build on cross- experiment collaboration Encourage new joint developments across experiments New areas are PhenoGrid and generic portal work

22. Tony Doyle - University of Glasgow Middleware, Security and Network [FTE] Information Services [4+4 FTE] and Networking [2+1.5+1.5 FTE]: strategic roles within EGEE Security expands to meet reqts. Storage and Workload Management continue No further configuration management development programme defined by –mission criticality (experiment requirements driven) –International/UK-wide lead –leverage of EGEE, UK core and LCG developments Redefine (EDG-based) UK Work Packages into Development Groups Metadata area requires group defined in association with applications area General emphasis on deployment/support rather than development per se Security Middleware Networking

23. Tony Doyle - University of Glasgow Tier-2 Services [FTE] Four Regional Tier-2 Centres London: Brunel, Imperial College, QMUL, RHUL, UCL. SouthGrid: Birmingham, Bristol, Cambridge, Oxford, RAL PPD. NorthGrid: CCLRC Daresbury, Lancaster, Liverpool, Manchester, Sheffield. ScotGrid: Durham, Edinburgh, Glasgow. Hardware provided by Institutes GridPP provides added manpower

24. Tony Doyle - University of Glasgow Tier-1/A Services [FTE] High quality data services National and International Role UK Focus for International Grid development Highest single priority within GridPP2 Total Manpower CPU 2.0 Disk 1.5 Tape 2.5 Core Services 2.0 Operations 2.5 Networking 0.5 Deployment 2.0 Experiments 2.0 Management 1.5 Total 16.5

25. Tony Doyle - University of Glasgow Deployment Board Tier1/Tier2, Testbeds, Rollout Service specification & provision User Board Requirements Application Development User feedback Metadata Workload Network Security Info. Mon. PMB CB Storage Proposed Structure

26. Tony Doyle - University of Glasgow In LCG Context Proposed Structure ARDA Expmts EGEE LCG Deployment Board Tier1/Tier2, Testbeds, Rollout Service specification & provision User Board Requirements Application Development User feedback Metadata Workload Network Security Info. Mon. PMB CB Storage

27. Tony Doyle - University of Glasgow GridPP2 User Board: Two Roles?? The role of Applications Manager(?) is to: 1.lead the Application Development Team; 2.sit on the User Board; 3.ensure the formulation and implementation of application middleware necessary to ensure timely deployment; 4.report to the User Board on progress and issues; 5.liaise with all the application experts on the technical implementation of the production grid. The role of the User Board Chair is to: 1.The role of the User Board Chair is to: 2.convene and chair User Board meetings; 3.sit on the Project Management Board; 4.inform the User board of issues from the PMB; 5.solicit input from users and potential users; 6.inform the PMB and DB of user requirements and issues; 7.ensure that progress towards deliverables in the application interface area is monitored on a quarterly basis; 8.report to the Project Manager on matters relating to any project deliverables and milestones defined within his or her purview. User Board Experiments

28. Tony Doyle - University of Glasgow General Points 1.All post-holders report back at 3-monthly intervals (using similar reporting mechanisms to those used for the majority of GridPP posts). Reporting through either the: 1.User Board; or 2.Deployment Board 2.All GridPP2 posts will end on Aug 31st 2007. Existing posts that are renewed will commence at the end-date of the present grant. 3.A review of posts will be made prior to this end point, in order to determine their effectiveness within the GridPP2 programme.

29. Tony Doyle - University of Glasgow Where does the work get done? Areas on Green Alert Approved subject to minor/moderate changes: timeline described by Steve… Groups bridge across the applications/middleware/support boundaries Active subgroups linking to EGEE: 1.Information Services –RAL LCG developments (GridPP2/LCG/EGEE joint programmes?) 1.Security inc. VO management Group Manchester, RAL [DK, AM] 2.Workload Management IC [DC] Deployment groups 1.Data Replication Group (support, small, but necessary) RAL, Glasgow, Bristol… 2.Others.. These can also be task-force like areas addressing specific issues.. Workshop organised by RM

30. Tony Doyle - University of Glasgow Areas on Amber Alert First discussions on not yet approved and approved subject to major changes areas We need to develop a detailed plan taking on board PRSC input not defined a priori 1.Metadata Group Glasgow, Oxford, IC/Bristol? ++? [TD] 2.Storage Management Group RAL, Edinburgh, Bristol [JG] 3.Networking Group UCL, DL, Mcr [PC] New deliverables in 1-2 months…

31. Tony Doyle - University of GlasgowOutcome? We need to specify management/technical roles in advance of GridPP2 to ensure we are successful in developing/deploying the Production Grid By the end of this GridPP2 specification process we will be able to: –Continue the work started in GridPP1 (by many people) –Define the GridPP2 Project Map tasks and metrics –Appoint additional people in recognised areas to contribute to agreed tasks/roles –Contribute fully to EGEE/LCG –Start to go from Prototype to Production Anticipate new management/technical posts and holders are defined before GridPP10 meeting at CERN in June Beyond this, we anticipate a transition after GridPP2 in the PPARC Road Map where these areas are supported within the Particle Physics Programme We are masters of our own destiny..