1. DataGrid is a project funded by the European Commission under contract IST-2000-25182 Status and Prospective of EU Data Grid Project Alessandra Fanfani (University of Bologna) On behalf of EU DataGrid project Outline:  EU DataGrid project  HEP Application experience  Future perspective http://www.eu-datagrid.org

2. The European DataGrid Project - n° 2The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 The EU DataGrid Project  9.8 M Euros EU funding over 3 years  90% for middleware and applications (HEP, Earth Observation, Biomedical)  3 year phased developments & demos  Total of 21 partners n Research and Academic institutes as well as industrial companies  Extensions (time and funds) on the basis of first successful results: n DataTAG (2002-2003) www.datatag.org n CrossGrid (2002-2004) www.crossgrid.org n GridStart (2002-2004) www.gridstart.org  Project started on Jan. 2001  Testbed 0 (early 2001) n International test bed 0 infrastructure deployed s Globus 1 only - no EDG middleware  Testbed 1 ( early 2002 ) n First release of EU DataGrid software to defined users within the project  Testbed 2 (end 2002) n Builds on Testbed 1 to extend facilities of DataGrid n Focus on stability  Passed 2 nd annual EU review Feb. 2003  Testbed 3 (2003) n Advanced functionality & scalability n Currently being deployed  Project stops on Dec. 2003

3. The European DataGrid Project - n° 3The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Related Grid Projects Through links with sister projects, there is the potential for a truly global scientific applications grid Main components of EDG 2.0 release build the basis for LCG middleware LHC Computing Grid www.cern.ch/lcg www.cern.ch/lcg

4. The European DataGrid Project - n° 4The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EDG Middleware Architecture Collective Services Information & Monitoring Replica Manager Grid Scheduler Local Application Local Database Underlying Grid Services Computing Element Services Authorization Authentication and Accounting Replica Catalog Storage Element Services SQL Database Services Fabric services Configuration Management Configuration Management Node Installation & Management Node Installation & Management Monitoring and Fault Tolerance Monitoring and Fault Tolerance Resource Management Fabric Storage Management Fabric Storage Management Grid Fabric Local Computing Grid Grid Application Layer Data Management Job Management Metadata Management Service Index APPLICATIONS GLOBUS CondorG (via VDT) M / W

5. The European DataGrid Project - n° 5The 2 nd Workshop on HEP GRID – Daegu 22 August 2003  The user interacts with Grid via a Workload Management System (WMS)  The Goal of WMS is the distributed scheduling and resource management in a Grid environment.  Resource Broker tries to match user requirements with available resources n Software installed at potential sites n Ensure data locality n Efficient usage of resources Workload Management System

6. The European DataGrid Project - n° 6The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Data Management  High level data management on the Grid n Location of data n Replication of data n Efficient access to data  Provide basic, consistent interface to disk and mass to storage systems (Hides the Storage Resource Manager )

7. The European DataGrid Project - n° 7The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Information & Monitoring  R-GMA Relational implementation of GMA from GGF  Makes use of GLUE schema (inter-operability with US grids)  Interoperable with MDS  Deals with information on n The Grid itself s Resources and Services s Job status information n Grid applications

8. The European DataGrid Project - n° 8The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Grid aspects covered by EDG VOMS (VO Membership Service) Provides certificate with VOs, groups and roles RGMA: Information & Monitoring Provides info on resource utilization & performance User Interface Submit & monitor jobs, retrieve output Grid Fabric Management Configure, installs & maintains grid sw packages and environ. Workload Management System Manages submission of jobs to Res. Broker, obtains information and retrieves output Network performance Provides efficient network transport, bandwidth monitoring Computing Element Gatekeeper to a grid computing resource Testbed admin. Certificate auth.,user reg., usage policy etc. Storage Resource Manager Grid-aware storage area Applications HEP, EO, Biology Replica Manager Replicates and locates data

9. The European DataGrid Project - n° 9The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Detailed Interplay of EDG Components

10. The European DataGrid Project - n° 10The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 People >350 registered users 12 Virtual Organisations 16 Certificate Authorities >300 people trained 278 man-years of effort 100 years funded Scientific applications 5 Earth Obs institutes 9 bio-informatics apps 6 HEP experiments DataGrid in Numbers Software 50 use cases 18 software releases Current release 1.4 Release 2.0 being tested >300K lines of code Testbeds >15 regular sites  40 sites using EDG sw (i.e. Taiwan, Korea) >10’000s jobs submitted >1000 CPUs >15 TeraBytes disk 3 Mass Storage Systems

11. The European DataGrid Project - n° 11The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 DataGrid Scientific Applications Earth Observation about 100 Gbytes of data per day (ERS 1/2) 500 Gbytes, for the ENVISAT mission Bio-informatics n Data mining on genomic databases (exponential growth) n Indexing of medical databases (Tb/hospital/year) Particle Physics  Simulate and reconstruct complex physics phenomena millions of times  LHC experiments will generate 6-8 PetaBytes/year Developing grid middleware to enable large-scale usage by scientific applications  Development on computing side but also focus on the real use by the applications!

12. The European DataGrid Project - n° 12The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Application Usage of Release 1.4 Positive Signs:  Large increase in users.  Many sites interested in joining.  Pushing real jobs through system. EDG 1.4 evaluated for review in Feb. 2003 CEs HEP Simulation Disk Usage CPU Usage CEsSEs Nb. of evts 1MB1MB 1GB1GB 1TB1TB TOTAL: >1.5 TB 100 GB 19 G B 200 GB Disk Usage (CERN) Successful 2 nd annual EU review: funding agencies were happy about the real use by the application

13. The European DataGrid Project - n° 13The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 HEP Applications  Intense usage of application testbed in 2002 and early 2003, in particular by HEP experiments:  ATLAS, CMS, ALICE, LHCb, Babar, D0 activities within DataGrid documented in detail in deliverable D8.3 https://edms.cern.ch/document/375586/1.2 https://edms.cern.ch/document/375586/1.2  ATLAS and CMS task forces very active and successful s Several hundred ATLAS simulation jobs of length 4-24 hours were executed & data was replicated using grid tools s CMS Generated ~250K events for physics studies with ~10,000 jobs in 3 week period n Since project review: ALICE and LHCb have been generating physics events n Babar and D0 performed more basic tests with analysis and Monte-Carlo production jobs

14. The European DataGrid Project - n° 14The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Joint evaluation from Atlas/CMS work on Release 1.4  Results were obtained from focused task-forces of Experiments and EDG people n Good interaction with EDG middleware providers n Fast turnaround in bug fixing and installing new software  Test were labour intensive since software was developing and the overall system was fragile  There are essential developments needed in n Data Management (robustness and functionality) n Information Systems (robustness and scalability) n Workload Management (scalability for high rates, batch submissions,stability) n Mass Storage Support (gridified support due in EDG 2.0)  Release 2.0 should fix the major problems

15. The European DataGrid Project - n° 15The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Release 2.0  Major new developments in all middleware areas  Addressing the key shortcomings identified: n WMS stability and scalability  WMS re-factored n Replica catalog stability and scalability  Replica Location Service n Data management usability  DM re-factored n Information system stability and scalability  R-GMA n Unified access to MSS  new SE service n Fabric monitoring infrastructure  Providing new functionalities  Upgrade underlying software

16. The European DataGrid Project - n° 16The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 HEP experience:the CMS example joint effort involving CMS, EDG, EDT and LCG people  CMS/EDG Stress Test Goals : n Verification of the portability of the CMS Production environment into a grid environment; n Verification of the robustness of the European DataGrid middleware in a production environment; n Production of data for the Physics studies of CMS  Use as much as possible the High-level Grid functionalities provided by EDG: n Workload Management System (Resource Broker), n Data Management (Replica Manager and Replica Catalog), n MDS (Information Indexes), n Virtual Organization Management, etc.  Interface (modify) the CMS Production Tools to the Grid provided access method  Measure performances, efficiencies and reason of job failures to have feedback both for CMS and EDG

17. The European DataGrid Project - n° 17The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 CMS/EDG Middleware and Software  Middleware was: EDG from version 1.3.4 to version 1.4.3 n Resource Broker server n Replica Manager and Replica Catalog Servers n MDS and Information Indexes Servers n Computing Elements (CEs) and Storage Elements (SEs) n User Interfaces (UIs) n Virtual Organization Management Servers (VO) and Clients n EDG Monitoring, etc…  CMS software distributed as rpms and installed on the CE  CMS Production tools (IMPALA,BOSS) installed on User Interface  Monitoring was done trough: n Job monitoring and bookkeeping: BOSS Database, EDG Logging & Bookkeeping service n Resources monitoring : Nagios, web based tool developed by the DataTag project n EDG monitoring system (MDS based): collected regularly by scripts running as cron jobs and stored for offline analysis n BOSS database: permanently stored in the MySQL database Both sources are processed by a tool ( boss2root ) to put the information in a Root tree to perform analysis On line Off line

18. The European DataGrid Project - n° 18The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 CMS jobs description CMKIN Job CMSIM Job Output data (ntuples) Output data (Fz files) Grid Storage Write to Grid Storage Element Write to Grid Storage Element Read from Grid Storage Element * PIII 1GHz 512MB  46.8 SI95 size/eventtime * /event CMKIN ~ 0.05MB~ 0.4-0.5 sec CMSIM ~ 1.8 MB ~ 6 min Dataset eg02_BigJets  CMS official jobs for “Production” of results used in Physics studies : Real-life testing  Production in 2 steps: 1. CMKIN : MC Generation of the proton-proton interaction for a physics channel (dataset) 125 events ~ 1 minute ~ 6 MB ntuples 2. CMSIM : Detailed simulation of CMS Detector 125 events ~ 12 hours ~ 230 MB FZ files “Short” jobs “Long” jobs

19. The European DataGrid Project - n° 19The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 CMS production components interfaced to EDG Four submitting UIs: Bologna/CNAF (IT), Ecole Polytechnique (FR), Imperial College (UK), Padova/INFN (IT) Several Resource Brokers (WMS), CMS-dedicated and shared with other Applications: one RB for each CMS UI + “backup” Replica Catalog at CNAF, MDS (and II) at CERN and CNAF, VO server at NIKHEF CMSEDG BOSS DB Workload Management System JDL RefDB parameters input data location Push data or info Pull info UI IMPALA/BOSS Replica Manager CE CMS software CE CMS software CE WN SE Job output filtering Runtime monitoring CE CMS software SE data registration read write SE CE CMS software X

20. The European DataGrid Project - n° 20The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EDG hardware resources Site Number of CPUs Disk Space GB Availability of MSS CERN (CH)122 1000 * (+100) yes CNAF (IT)20 + 20 * 1000 * RAL (UK)16360 Lyon (FR) shared 120 (400) 200yes NIKHEF (NL)2235 Legnaro (IT) * 501000 * Ecole Polytechnique (FR) * 4220 Imperial College (UK) * 16450 Padova (IT) * 12680 Totals402 (400) 3000 * + (2245) * Dedicated to CMS Stress Test CNAF Bologna Legnaro & Padova CERN Ecole Poly RAL. Imperial College NIKHEF Lyon add new (CMS) sites to provide extra resources

21. The European DataGrid Project - n° 21The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Statistics of CMS/EDG Stress Test Nb of jobs Executing Computing Element Total EDG Stress Test jobs = 10676, successful =7196, failed = 3480 Total nb. of events CMKINCMSIM 592750268375 Total size of data produced  500 GB distribution of job: Executing CEs

22. The European DataGrid Project - n° 22The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 CMS/EDG Production ~260K events produced ~7 sec/event average ~2.5 sec/event peak (12-14 Dec) 30 Nov 20 Dec CMS Week Upgrade of MW Hit some limit of implement. (RC,MDS) CMSIM “long” jobs Nb of events job submitted from UI:

23. The European DataGrid Project - n° 23The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 Main results and observations  RESULTS n Could distribute and run CMS software in EDG environment n Generated ~250K events for physics with ~10,000 jobs in 3 week period  OBSERVATIONS n Were able to quickly add new sites to provide extra resources n Fast turnaround in bug fixing and installing new software n Test was labour intensive (since software was developing and the overall system was fragile) s WMS: At the start there were serious problems with long jobs- recently improved s Data Management: Replication Tools were difficult to use and not reliable, and the performance of the Replica Catalogue was unsatisfactory s Information system: The Information System based on MDS performed poorly with increasing query rate s The system is sensitive to hardware faults and site/system mis-configuration s The user tools for fault diagnosis are limited n EDG 2.0 should fix the major problems providing a system suitable for full integration in distributed production

24. The European DataGrid Project - n° 24The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EU DataGrid Summary and Outlook  The focussing of the project on stability has improved the manner in which the software is build and supported  The application testbed has reached the highest level of maturity that can be achieved using the available grid middleware and supporting manpower n Steady increase in the size of the testbed until a peak of approx 1000 CPUs at 15 sites n Intense usage of application testbed (release 1.3 and 1.4) in the past year significant achievements in the use of EDG middleware by the experiments : s Real use is possible but labour intensive s Results were obtained by task-force which pointed to areas in the middleware which required development and reconfiguration  The problems in performance encountered by the experiments are addressed in the release EDG 2.0.  There is a strong connection with the LHC Computing Grid. LCG have a new grid service modeled on the EDG testbed and includes EDG 2.0 components Outlook: A production quality infrastructure is needed  EGEE Continuous, stable Grid operation represents the most ambitious objective of EGEE and require the largest effort

25. The European DataGrid Project - n° 25The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EGEE vision: Enabling Grids for E-science in Europe  Goal n Create a wide European Grid production quality infrastructure on top of present and future EU RN infrastructure  Build on n EU and EU member states major investments in Grid Technology n Exploit International connections (US and AP) n Several pioneering prototype results n Large Grid development team (>60 people) n Requires major EU funding effort  Approach n Leverage current and planned national and regional Grid programmes (e.g. LCG) n Work closely with relevant industrial Grid developers, NRENs and US-AP projects EGEE Applications Geant network http://www.cern.ch/egee

26. The European DataGrid Project - n° 26The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EGEE Proposal  Proposal submitted to EU IST 6 th framework call on 6th May 2003 n Executive summary (exec summary: 10 pages; full proposal: 276 pages) http://agenda.cern.ch/askArchive.php?base=agenda&categ=a03816&id=a03816s5%2Fdocu ments%2FEGEE-executive-summary.pdf  Two-year project conceived as part of a four year programme 9 regional federations covering 70 partners in 26 countries

27. The European DataGrid Project - n° 27The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EGEE Operation Management Regional Operations Centre Core Infrastructure Centre  Service Activities: deliver production level Grid Infrastructure (52% of funding) n Integration of national and international Grid infrastructures n Essential elements: manageability, robustness, resilience to failure,consistent security model, scalability to rapidly absorb new resources  Joint Research Activity: Engineering development (24% of funding) n Re-Engineering of grid middleware (OGSA environment) to improve the services provided by the Grid infrastructure  Networking Activities:Management, Dissemination, Training and Applications (24% of funding) n The Applications Interface Activity will start with two Pilot applications in high energy physics and bio/medical EGEE Activities  managing the overall Grid infrastructure  regional deployment and support of services

28. The European DataGrid Project - n° 28The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EGEE Status  EGEE proposal passed thresholds at first EU review (June 2003) n Follow-up hearing held at Brussels on 1 st July 2003 to answer written questions from the EU reviewers on details of the project  Evaluation Summary Report received from Brussels (17 th July 2003) n Number of detailed recommendations made EU budget estimated at 31.5M €  Negotiate budget details during summer and produce Technical Annex (details of negotiated tasks and budgets) s Informal EGEE/EU meeting held in Brussels 24 th July 2003  Foreseen project start date: 1 st April 2004 Good match with existing EU DataGrid and related project expected completion All partners are requested to assign resources already during summer 2003 to start engineering investigations and architecture design work so that project can start on time

29. The European DataGrid Project - n° 29The 2 nd Workshop on HEP GRID – Daegu 22 August 2003 EGEE Summary  EGEE is a project to develop and establish a reliable infrastructure that provides high quality grid service to a wide range of users  HEP is one of the two pilot application areas selected to guide the implementation and certify the performance and functionality of this evolving European Grid infrastructure  International connection : participation and collaboration with non EU countries (Russia, US, AP) is desirable and will be pursued