1. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-1 The INFN Grid evolution and the coordination of the experiments common computing activities Perugia – November 2002 Mirco Mazzucato INFN-Padova mirco.mazzucato@pd.infn.it

2. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-2 Introduzione  L’ obiettivo primario del Progetto Speciale INFN-Grid, come descritto nella proposta, approvata agli inizi del 2000,  http://www.pd.infn.it/~mmazzuca/infn grid v20final  “consiste nello sviluppo e attuazione comune di un prototipo di sistema di calcolo, basato sul middleware di GRID e finalizzato a rispondere alle esigenze degli esperimenti futuri con particolare riguardo ad LHC”  Al momento della proposta il progetto Datagrid stava per essere proposto alla EU, i singoli esperimenti avevano i loro propri modelli di calcolo e mancava totalmente una visione unitaria di come impostare il calcolo per LHC.  Il quadro attuale della situazione e’ notevolmente cambiato e in questo talk cerchero’ di descrivere come il progetto sta evolvendo per permettere ai gruppi italiani di essere competitivi nell’analisi a LHC e nello stesso tempo partecipare all’impresa piu’ vasta della creazione di una Grid mondiale per la scienza

3. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-3 Summary  The INFN Grid project  The foundation: Farms and Condor pools  Evolution of participants and activities  The international projects  LCG  The INFN development of middleware  The new initiatives  The Italian eScience activities: The FIRB project  The new IG-BIGEST initiative  Areas of activities  Participants  The EGEE proposal  Conclusions  The new INFN Grid management structure

4. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-4 The 90’s revolution…  Linux Farms become available in many Computer Centers, Universities, Research Centre and Industries (ISP, WEB Portals…) A lot of experience on them.  Basic hardware is constituted by commodity components which are easy to find, manage and that can provide very cheap large computing and storage capacity in a very limited space  Technology Tracking is reassuring on continuous improvements (at least up to 2006-2007) in both density and power Now:200 CPU Blade boxes or 200 disks in a standard rack 60x80 cm 2 ; CPU and storage power will continue to grow for same cost according to Moore’s law (x2 every 14-18 months) WAN is now a commodity thanks to opening of the markets: Moore’s law : x2-4 bandwidth increase/year at equal cost 10 Gbps available now in WAN (Garr, Geant evolution)  Hardware and connectivity are not a problem for the growth and access to large computing facilities ( ….for at least other 4-5 years). The real issue for Science and LHC experiments in particular is to be able to exploit them efficiently to reach their physics goal

5. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-5 The ’90s Foundation:The first INFN Computing GRID based on Condor INFN has a long tradition of excellence on distributed commodity computing e.g. in experiments  DELPHI INFN Farm in ’89, L3 simulation Farm in Naples.... ..and computing services and CNAF  INFN Project “Condor on WAN” aiming to configure Condor batch system on most of the INFN computers distributed over the Italian Research network to provide transparent access to CPU and data  Collaboration between INFN and the Condor Team of the Computer Science Department of wisconsin-Madison University.  www.infn.it/condor/ www.infn.it/condor/  Could be still operational as a Grid CE to use spare cycles  The weakness on Security (AAA),lack of heterogeinity, flexibility, VO mangement, data management... brought to the Grid concept but  the experience gained with INFN Condor pool was unique in the world  CNTC allowed to gain experience in common projects: Monarc, Geant4…  Garr is providing efficient connections to all Italian Research sites

6. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-6 GARR-B Topology 155 Mbps ATM based Network access points (PoP) main transport nodes TORINO PADOVA BARI PALERMO FIRENZE PAVIA MILANO GENOVA NAPOLI CAGLIARI TRIESTE ROMA PISA L’AQUILA CATANIA BOLOGNA UDINE TRENTO PERUGIA LNF LNGS SASSARI LECCE LNS LNL USA 155Mbps T3 SALERNO COSENZA S.Piero FERRARA PARMA CNAF Central Manager INFN Condor Pool on WAN: checkpoint domains ROMA2 10 40 15 4 65 5 Default CKPT domain @ Cnaf CKPT domain # hosts 10 2 3 6 3 2 3 5 1 15  machines  500-1000 machines 6 ckpt servers  25 ckpt servers Slide from A.Ghiselli

7. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-7 INFN Grid: participants  INFN Grid was including since the beginning:  The computing professional of most INFN sites  The 4 LHC experiments  Virgo  APE (the INFN home developed parallel computing for Lattice QCD)  Several groups of computing scientists: Catania, Torino, Pisa, Venezia, Bologna...  ~50 technical FTEs + experiments  It has been the first common project for HEP computing approved in the world  UK GridPP: end of 2000  LCG: end of 2001  Now: common US ATLAS, CMS proposal under discussion in LBL  It has been enlarged recently to CDF and Babar….

8. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-8 LCG

9. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-9 INFN Grid: management  It has been managed by  the Executive Board which included  the computing coordinators of the LHC experiments and Virgo  APE (not very active)  The INFN Grid technical coordinator  The Director of Cnaf INFN representative in DataGrid  The project Manager  The Technical Board constituted by  the Managers of the WP or leader of most important area of activities(e.g. Security, Genius portal..)  Site administrative coordination is provided by Colgs  It has proven to be an efficient structure but need to evolve to face the evolution of activities

10. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-10 INFN Grid: activities (traditional) Having always on focus the LHC computing needs INFN Grid has developed activities on all those components of the computing model which allow in principle common solutions. These tradionally included:  The development of the National testbed (including Tier2,3 prototypes integrated with the Tier1) and related services  DataGrid: development of the grid middleware and of the European testbed  DataTAG: Grid M/W interoperability with US and creation of the first World Grid testbed based on de facto standard components  Grid integration in experiments WP4.4  The INFN participation to internationl coordination bodies  HEP Joint Technical Board (JTB)  Originate common WG EU-US (GLUE)  HEP Intergrid Collaboration Board (HICB)  The contributions to the Grid standardization activities in the Global Grid Forum (GGF)  INFN M/W specific developments: Monitoring (Nagios), Genius portal, LFCG installation kit, test suites....

11. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-11 INFN Grid: activities (new) .INFN Grid is now faced to coordinate also  The common INFN contribution to LCG and in particular the Italian prototype LCG infrastructure deployment  Provide common INFN interface to common LCG activities  The FIRB eScience project( recently approved) extending the INFN Grid to the other Sciences  The new round of EU Grid projects for the Framework Program 6th (FP6, starting at beginning 2003) aiming at developing an European Grid Integrated Infrastructure Initiative called “Enabling Grid and Escience in Europe” (EGEE)  The Italian Grid- for Business Industry Government EScience & Technology initiative (IG-BIGEST) which organize the Italian participation to the construction of the European Grid Infrastructure by EGEE of all Science Institutions and many Industries and is coordinated by INFN  The goal is to develop maximal sinergies in INFN and share/recycle expertise and efforts between different projects

12. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-12 INFN Grid: what is not included ?  The development of the Tier1 prototype at CNAF  Responsibility of CNAF director and related structures  The development of experiments applications  Framework, test beams analysis, detector testing, development of reconstruction, simulation and analysis programs  The experiments participation to the Application area of LCG common projects (RTAGs)  Persistency  Detector description  Simulation ................ ....not much INFN presence yet  The experiments specific production activities not LCG

13. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-13 DataGrid: activities www.edg.org  Financed by EU for Middleware development and testbed (9.8 M€)  INFN: 1 M€, CNR: 0.3 M€, Datamat 0.5 M€, IRST(Tn): 0.3 M€  Italy: 2.1 M€ for personnel for 3 years  Hired 11 FTEs for INFN  The organization of  WP1 (Workload Management)  WP2(Data Management)  WP4( Fabric Management, LCFG..)  WP8 (HEP applications)  will remain unchanged until completion of the project at the end 2003  Particular care will be devoted to guarantee support to WP1 manged by INFN  INFN will start an “official” unfunded activity on WP3( Grid Monitoring) to capitalize the developments done so far internally (Nagios)  WP6(testbed) will be completely reorganized to take into account the new activities:  EDG will provide basic middleware, prototype testbed and services for LCG-1

14. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-14 DataGrid: INFN responsibilities  Project Management Board :  Federico Ruggieri (rappresentante INFN) and Mirco Mazzucato  Project Technical Board :  Paolo Capiluppi (CMS), Piergiorgio Cerello (ALICE), Laura Perini (ATLAS) Francesco Prelz (WP1), Massimo Sgaravatto (WP1 deputy), Antonia Ghiselli (WP6 deputy)  WP1:  Francesco Prelz (manager), Massimo Sgaravatto (deputy)  WP6:  Antonia Ghiselli, deputy manager

15. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-15 DataTAG and GLUE: activities www.cern.ch/datatag  Financed by EU with 0.6 M€ for personnel  All people hired in a very short time (10FTEs)  WP4 (Grid interoperability EU-US) started very fast  In few months INFN, as EDT WP4 leader, was already actively starting with the US project iVDGL, the GLUE ( Global Laboratory Uniform Environment) http://128.3.182.66/mtgs/ivdgl/Interoperability.htm activities covering almost all topics relevant to solve Grid interoperability issues between EU and US: http://128.3.182.66/mtgs/ivdgl/Interoperability.htm  Middleware  Services  Experiment Grid integration support  INFN has a small participation in WP2 (Network measurements)  Common management of EDG and EDT provide remarkable synergy  The development of common solutions (or at least agreed and introperable) is a natural interest of LHC (or generally HEP) experiments, in particular for what concern the EU-US collaboration.  Most of the HEP experiments have strong US components together with EU ones  INFN DataTAG with iVDGL have provided the first interoperable EDG(EU) and VDT(US) middleware for ATLAS and CMS  Demo at IST2002 an SC2002

16. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-16 DataTAG and GLUE:responsibilities  Project Management Board EDT:  Mirco Mazzucato e Antonia Ghiselli  WP4 EDT (Interoperbility):  Antonia Ghiselli manager WP4 Cristina Vistoli (Deputy)  GLUE  Antonia Ghiselli (EU) coordinatore con Ruth Pordes (iVDGL- US) delle attivita’ di interoperabilita’  Cristina Vistoli (deputy)  Flavia Donno responsabile integrazione applicazioni

17. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-17 The development of the national testbed and the INFN development of middleware www.infn.it/grid  The testbed is progressively incorporating all INFN sites. No exclusions but encouradgement to provide the necessary local efforts  Progress on middleware developed by INFN Grid was unexpected  Genius Portal is becoming almost a standard  Used in all demos and being evaluated by several experiments: ATLAS, CMS,.....  Monitoring with Nagios is expanding and becoming an official EDG WP3 activity and considered by LGC  LCFG configuration kit is now the baseline for Datagrid  The Test Suite released by INFN has been adopted by LCG  their development should contine and their usage by experiments or in a wider eScience context should be encouradged  Tutorial and support should continue and extended to other Sciences  Next Grid tutorial organized in Torino December 5th and 6th(L.Gaido)  INFN teams inside LHC experiments are promoting the usage of these INFN Grid developed components in experiment and LCG

18. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-18 The Gridification of the INFN resources  They consists of  Dedicated resources for middleware testing&validation  6 sites with a large cluster ( large number of CPUs and large disk server)  13 sites with smaller clusters  Tier1 prototype at CNAF (INFN National Computing Facility)  3 LHC Tier2 prototypes ( ~50 processors and 5TB each)  Virgo, CDF and Babar resources (Hundred of PCs and TB of disks)  Grid INFN sites now cover the whole Italy

19. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-19 InfnGrid & DataGrid testbeds EDG Appl. & partners VOs INFN-Grid VO Infn sites Originally 2 grid domains: 5 sites for DataGrid Testbed1 5+14 sites for national Testbed1 CERN Lyon/in2p3 RAL/pparc nikhef mito ba ca na le pd/lnl Cnaf/bo ct roma ts ge pv rm3 fe pr Each Infn Grid site can easily join any Testbed: e.g. LCG EU sites Now: many, many testbeds pi

20. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-20 The INFN Infrastructure for LCG (1)  The deployment of the LCG infrastructure is organized by LCG for Tier1 and 2  Ramp up foresees to start with 5-6 major centers in June 2003 and incorporate progressively all Tiers 1-2 (20) at the end of 2003  GDB Meeting 4/11/02 referring to RC resources:  “In particular it has been clarified that both the commitment to LHC and the fraction committed to LCG will be indicated. In principle this should apply to the 2003 transition phase. The fraction is supposed to go to 1 from the 2004 onwards “  Near to LCG production infrastructure will coesist a development one (as in EDG)  The LCG infrastructure must provide theTier1 and Tiers2 services to experiments 24hx24h x7

21. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-21 The INFN Infrastructure for LCG (2)  INFN will present at the international level only the global infrastructure Tier1+Tiers2 to have more weight. See e.g. Paolo&Laura  LCG resources will be shared by experiments, but with definite piorities. The same is proposed for the INFN LCG infrastructure  Resources are in general allocated to experiments and these maintain control on use policies.  Expts also agree to share with policies to be defined by GDB.  In INFN Tier3 play an important role. They certainly should contain a flexible User Interface to the LCG infrastructure.  Since Tier3 are not included in the LCG activities, INFN Grid has to take care of their set up and integration with the Tier1-2 LCG infrastructure:  -> New INFN Grid working group eventually extended at international level

22. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-22 The INFN Infrastructure for LCG (3)  WP6 (testbed) of INFN Grid will coordinate the deployment and operation of the INFN LCG infrastructure. It includes already:  Experts and site managers for the deployment of the middleware and deployment/operation of the basic Grid services  Grid and middleware experts according to the needs  Will now include  Responsible of experiments who assume a steering role  WP6 will define the sub-teams, e.g. INFN Grid LCG deployment group, support group, operation group, etc, to correctly interface similar LCG structures at CERN  WP6 will monitor and guarantee global operation and availability day by day of INFN LCG resources in this prototype phase  WP6 will be jointly managed by a Grid technical expert and an experiment representative  INFN LCG must be made available to LHC experiments according to the milestones of the High Level Plan (Preliminary Status)  http://lhcgrid.web.cern.ch/lhcgrid/peb/document/21jun02_LCG_p lanning_status.pdf (see Masoni talk) http://lhcgrid.web.cern.ch/lhcgrid/peb/document/21jun02_LCG_p lanning_status.pdf  Start of LCG production service: June 2003  Full operation 7x7x24 ( >99% efficiency): January 2004

23. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-23 Priorities for INFN LCG  To present a common interface to the LCG infrastructure it will be necessary to make a global planning of the INFN resources  It is proposed that this is taken care by the current Tier1 “Comitato di Gestione” extended to include Tier2 representatives  It is proposed that priorities in the usage of INFN LCG infrastructure be:  LHC Experiment X for its allocated resources  The other LHC experiments (priority 2) since they are part of the common shared LCG infrastructure  The other experiments Babar, Virgo and CDF....... (Priority 3)  Other users (Priority 4)

24. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-24 INFN Responsibilities in LCG  Project Executive Board (PEB):  Mirco Mazzucato (rappresentante INFN e Grid Deployment Board ),  Alberto Masoni (ALICE)  Software Computing Commitee (SC2):  Paolo Capiluppi (CMS) e  Federico Ruggieri (rappresentante Tier-1 INFN)  Grid Deployment Board (GDB):  Mirco Mazzucato (chair),  Federico Ruggieri (rappresentante INFN ),  Luciano Gaido (deputy del rappresentante INFN e responsabile INFN LCG infrastructure),  Alberto Masoni (deputy ALICE e segretario),  Laura Perini (deputy ATLAS)

25. last update: 03/06/2015 12:29 Mirco Mazzucato Infn -Padova 25 From INFN Grid to an Italian Grid: www.pd.infn.it/bigest The FIRB Project and The Italian Grid for eBusiness eIndustry eGovernment EScience and Technology (IG-BIGEST) Initiative FIRB: Fondo per gli Investimenti della Ricerca di Base(fondi MIUR da UMTS)

26. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-26 The Grid FIRB Proposal  Promoted directly by the INFN management  The motivation: INFN is well funded and has a long tradition of successful international collaborations  Should help other sciences to reach level of excellence and global collaborations sharing knowledge and top level enabling infrastructures like GRID  Governments in EU and US expects in general larger attention to technological fall out from HEP experiments  In FIRB INFN will provide the Grid technology know how and will receive personel from FIRB MIUR funds devoted to other Sciences to study and prototype a national Grid Operation Center (GOC) in Italy  The generalization of the infrastructure support is a model successfully established with the research network (GARR, GEANT..)  GOC will support extension of the INFN Grid to other Sciences and the operation of this Italian infrastructure  LCG will count possibly on general national GOCs (like the FIRB or UK and US ones) to guarantee 24x24x7 operations of the LCG infrastructure as counts on GEANT and GARR for the network

27. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-27  Proposals submitted in October 2001 to Ministry of Education University & Research (MIUR) Title of the proposal “Enabling Platforms for High-Performance Computational Grids Oriented to Scalable Virtual Organizations” Strategic Programme on Enabling IC Technologies for Knowledge Society Objectives and responsibilities for INFN:  Some R&D Grid technological development project complementing EDG (WP5)  Security, Data Management  Deployment of an Italian eScience Grid infrastructure (WP3)  Astrophysics  Biology  Computational Chemistry  Geophysics  Earth Observation………..  International integration by INFN (EDG,EDT,US) The FIRB eScience Grid in Italy

28. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-28 PISA NAPOLI COSENZA PERUGIA PADOVA GENOVA LECCE MILANO PALERMO ROMA TORINO MATERA PAVIA BARI BOLOGNA National Program Coordination CNR & University HPC, Parallel Programming, Grid computing, Scientific libraries, Data base and knowledge discovery, Earth Observation, Computational chemistry, Image processing, … ASI Applications of Earth Observation CNIT Technologies and HW-SW infrastructures for high-performance communication, Optical technologies, … CAGLIARI INFN & University Grid ( INFN-Grid, DataGrid, DataTag ), e-science applications: Astrophysics, Bioinformatics, Geophysics, … Cost 22,6 MEuro (7,2 MEuro co-founded), 33% of budget reserved for contracts to young researchers Funding: €8.1 M 3-Year Project Start-up: November 2002 UR INFN: €1.58M

29. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-29 Scopi del WP3 e WP5  applicazioni  - tools, frameworks, environment  computing system  global grid service Goal – come organizzare e gestire un prototipo di grid per la comunita’ della ricerca italiana WP3 non e’ un WP per lo sviluppo di un’altra tecnologia di grid ma e’ un WP per grid deployment Molto simile a LCG

30. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-30 Att.2 Service selection Att.2 Service selection requirements (a.1) Deployment (a.5) operations Deployment (a.5) operations Fabric e porting Fabric e porting FIRB-Grid Organizations Testbed sites iVDGL (VDT) DataGRID INFN-Grid DataTAG Globus Grid middleware Projects WP3,5 e le sue interazioni Packaging a.4 Packaging a.4 adaptation a.3 adaptation a.3 Firb-middleware LCG Integr.

31. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-31 Modello di Grid FIRB INGV Risorse E servizi Grid comune INAF Risorse E servizi Bio Risorse E servizi INFN-LCG Risorse E servizi Risorse E servizi comuni Altri Risorse E servizi

32. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-32 Gruppi per il grid deployment INAF INGVBio INFN WP3,5 planning WGs - Management architecture and tools - Autentication & Authorization group - Monitoring group - Deployment planning Grid Operation Center WGs Central management group Site managers User support SW packaging and documentation SW distribution Altri

33. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-33 Partecipazione e manpower  Organizzazione in corso di definizione:  Deployment planning:  1 persona per org.  Integrazione delle nuove risorse in grid  VO services  1 persona per org.  Grid operation/planning WGs  Management centrale : 2-3 persone  Management per sede : 1 persona per sede  User support : almeno 3 persone  SW distribution : 1 persona  Documentazione : 1 persona  Da reclutare per WP3-5 UR INFN: 9 persone: 3 art23, 6 art 2222

34. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-34 INFN responsibilities in FIRB  Mirco Mazzucato responsabile UR INFN,  Antonia Ghiselli (deputy) e rappresentante INFN nell’ UR,  Laura Perini, rappresentante applicazioni  Cristina Vistoli WP manager del WP3 e WP5  9 FTEs to be hired

35. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-35 Some issues  Grid M/W is being developed by several HEP projects:  in US: PPDG, GriPhyN, iVDGL,  in EU: DataGrid, DataTAG, Cross Grid...  and many others non HEP: Globus, Condor, Gridlab....  Grid M/W is adopted by a variety of Science sectors  Current M/W software is typical of prototype development. It is working but it is far from having well engineered production quality assuring long term survival and easy evolution  See Globus issues  The Grid middleware is deployed in several testbeds. This requires large manpower, duplications etc and all national Grid programs are looking for sinergies and common deployment and operation support  Policies for resource sharing within comunities having common objectives (VO), security issues, accounting etc start to be generally addressed only now with LCG  Grids are now seen by many governments as a Science development enabling factor and large funds are made available

36. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-36 Million Now: US proposal for research cyber-infrastructure: 1020 M$ Japan Grid: 500M$ EU FP6: 300 M€ for Geant and Grids

37. last update: 03/06/2015 12:29 Mirco Mazzucato Infn -Padova 37 The Italian Grid Initiative for eBusiness eIndustry eGovernment EScience and Technology (IG-BIGEST) http://www.pd.infn.it/~mmazzuca/bigest-fin-5-6-02.doc

38. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-38 eScience, eIndustry and the Grid  Why governments support Grids?  Modern fundamental Science in all domains requires more and more global collaboration (eScience) : improve efficiency, avoid effort duplication…..  eScience need new advanced support infrastructures  Modern Virtual Organizations,i.e.eScience teams of scientists with a common scientific goal, share resources, data, instruments and knowledge in real time at national, regional (Europe) and international level to make progress more efficient and faster  Modern Industries rely more and more on innovation which is increasingly based on the cycle:  Theoretical description of the problem  Simulation/Calculations of different options  Selction of best solution  Realization  and require access to large computing resources and collaborative methods  The Grid middleware aims to make available the services required by the e- Science and e-Industry Grid Application Projects  Italy spend 1% of GNP in research. With Grids it will share much larger efforts  Italy is and aims to remain a main player in the construction of the eScience world and the supporting Grid infrastructure

39. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-39 The Italian Grid for Business, Industry, Government, EScience&Technology (IG-BIGEST)  New initiative leveraging from MIUR proposals and aiming at extending the common development of the FIRB infrastructure to all Italian Sciences (e.g. INFM…) and open it to Industry early tests  Integration of the Grid Italian infrastructure in a European wide one is pursued through a common European Grid Integrated Infrastructure Initiative Project  "ENABLING GRIDS AND E-SCIENCE IN EUROPE " EGEE  Meeting in Rome (Presidenza INFN) on April 26 with the participation of all major Italian Scientific National Research Institutions, Universities and Computing Centres (~50) confirmed  General cosensus to foster the large European Initiative EGEE  CERN as leading partner and  INFN as organization coordinating the Italian Grid eScience comunity

40. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-40 Why EGEE  National Grid programs now exist and are increasingly developing Grid infrastructures in all EU countries  They are in general leaded by HEP but many science sectors are joining: Astrophysics, health and bio-medical, chemistry..  M/W is based on common de facto standard components  There has been an easy convergence on the following ideas:  to prepare a new EU Integrated Project to provide resources for integration, set up and operation of these national grids (like INFN-FIRB, UK..) into a common Grid EU infrastructure open to Business, Industry & Government  … and that LCG be the foundation of EGEE for all EU partners (including other Sciences). LCG is providing the first World Grid. This is generally recognized and consequences accepted

41. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-41 The participants and the Italian Grid eScience infrastructure  High bandwidth network provided by Garr-G -> Geant  Basic high performance computing infrastructures provided by National Research Institutions, Universities and Computing Centers :  Italian National Research Institutions  INFN, CNR, INAF, INFM, INGV……  Computing Centers  Cnaf, CINECA, CILEA, CASPUR…….  Universities  Università di Lecce, Università di Napoli, Università di Pisa, Università di Roma Tor Vergata, Università dell’Aquila, Università della Calabria, Università di Genova, Università di Padova, Politecnico Torino, Università di Salerno,Università di Messina,Universita’ di Milano....  Formal letter of participation have been received  See www.pd.infn.it/bigest

42. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-42 Areas of activity for IG-BIGEST IG-BIGEST partners provide top level expertise in all areas of activity  Grid Technology development  Creation and support of the eScience Grid Italian Centers  Deployment of the Italian Grid eScience infrastructure  Leveraging from INFN Grid infrastructure and test beds and Grid activities for Earth Observation and ASI in ESA-ESRIN Frascati, ENEA and University of Lecce (ISUFI)  And from the prototype Grid Operation Center in CNAF financed by MIUR  Development of specialized Grid Services with IT Industries( e.g. Datamat, Nice (Genius) already involved in EDG)  Development of Grid-enabled Applications for large Industries or SMEs (local and regional initiatives)  Grid application solutions for eGovernment eBusiness and eLearning  Dissemination and Industry and eScience Outreach  Italy can compete at level of excellence in all the above fields

43. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-43 Grid Technology development  Programming Tools and Environment  Security: AAAA in particular Audit and Accounting  Aim at the development of services that will bring current Grid security suitable for Business, Industries and Government Virtual Organizations  Data Intensive Services  Extension of current Data Grid Services to cover requirements of extended classes of Science  Knowledge discovery Services  Grid Portals  Resource Management (Brokering, Scheduling and Monitoring)

44. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-44 An important goal: Creation and support of EScience Italian Centres  IG_BIGEST will promote the creation of Italian eScience centres to support the development of the eScience Grid infrastructure  eScience centres (as in UK now)are distributed on major Italian sites and provide:  Grid technology R&D  Grid middleware services expertise  Grid infrastructure support  Spin off on Grid technologies and applications  Local forum and support for eScience :  HEP  Astronomy  Geophysics  Computational Chemistry  Biology  Earth Observation  Nanotechnologies  ………….  and BIG Grid applications  A National Center support the local Centers, provide international integration and expertise and foster extension to business, industry and government.

45. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-45 The EGEE Objectives and LCG (Preliminary agreement of partners and Les 5-11-2002)  The new EGEE project is conceived as being the creator and operator of the production EU Grid infrastructure  http://www.pd.infn.it/~mmazzuca/EGEE-EoIFinal0606.doc http://www.pd.infn.it/~mmazzuca/EGEE-EoIFinal0606.doc  EGEE should provide  Adaptation of middleware for HEP, bio-medical, other sciences  Re-engineering of basic M/W  Solve Globus and other M/W issues with EU engineered implementation  Advanced M/W  Operation of basic Grid infrastructure: 4-5 major general Grid Operation Centers and several National operation centers  Information and catalogues services  Coordination and operation  Call centre, user support  Training  Grid resources for physics, EO, biology, medicine… :Mainly unfunded  LCG will be in operation in 3 years and will be a model for the other sciences and main component of EGEE

46. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-46 The 1rst EGEE Proposal  Starting date of next FP6 call is December 17th (but not yet signed)  Call for Integrated Infrastructure Initiative  45-47 Millions available  1rst EGEE request: ~35 M€  Prliminar cost sharing  Overall management: 10%  Interface to applications: 10%  Hardware infrastructure 10%  Middleware re-engineering: 35%  Grid deployment and operation centers: 35%  1rst EGEE grid size: ~50 major sites, 5 call centers, minor sites  EGEE aims at bidding for ~100M€ with successive calls (3 in total)  Strong cooperation with Geant and successor

47. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-47 EGEE Partners Preliminary  Main Partners acting as coordinator of national grid activities  In general coordinating partners are from the HEP world  CERN: project coordinator  Fabrizio Gagliardi: Project Manager  UK, PPARC, Ireland  Germany, (Karlsruhe-Tier1)  Italy, INFN  France CNRS  Spain  Holland & Scandinavia  Czech Rep, Cesnet,  Poland  Russia  Hungary ..................

48. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-48 Why IG-BIGEST support EGEE poposal of a large general EU Grid infrastructure in FP6  Avoid divergence in Grid-Services(Currently tens of Grid EU and Nat. Projects..)  See US I-WAY and MPI  Favor eScience type collaboration in all Sciences (according to their assimilation time..) Very easy to set up EU VO (e.g. a HEP expts) if general EU infrastructure  Provide resources, skills, collaborative environments to allow EU Science to keep the level and compete with US and AP  Unify the debate between the R&D Grid development comunities and applications  Leverage and integrate developments from ongoing FP5 and National Projects  Put together all Grid rel. computing scientists, engineers and applications  Identify common requirements for Grid Services  Share view of priorities and strategies for Grid Service development and deployment

49. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-49 Why IG-BIGEST support EGEE....  Allow common interface to US and AP Grid initiatives  Favor selection of a common robust production Grid toolkit to be deployed in EU Grid Infrastructure(grant interoperability, open standars)  Initially more Grid Service implementations in EGEE toolkit but sites will force convergence  Separate infrastructures = separate toolkits  Build a multi-Terascale EU facility integrating all current major EU resources in all countries.  EU funds can extend and favour sharing of national resources to early EU users in all Sciences and Industries  Avoid initial billing and favor VO Agreements and MoUs on shared EU infrastructure

50. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-50 How to proceed IG-BIGEST and EGEE National Partners are now ready to:  Select and engineer Robust Grid Middleware components forming the first version of the Grid EU EGEE toolkit based on the requirements of a set of reference EU wide applications;  Identify and plan EU development of missing components e.g. general programming fmwk, security, accounting, monitoring, ….  Plan and identify National resources, sites, usage policies of Grid EU infrastructure  Set up EU wide authentication, authorization, registration and accounting policies and related services  Define and set up Grid operational model and procedures including user and application support  Support development of reference EU wide and International grid aware applications for the scientific and business sectors  Procure hardware components including specialized network where needed  Interface to ongoing International Grid efforts: EU, AP  Standardize in GGF type organizations  Grid national initiatives are ready to create an EU wide Grid pervasive infrastructure.

51. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-51 Summary  INFN Grid is successfully developing all major components of the LHC (and HEP) experiment computing infrastructure and is taking part at the same time to international efforts for the creation of a World Grid  As EDG main partner is providing the first EU M/W taylored to LHC needs  As EDT main partner and GLUE coordinator is providing interoperability with VDT(US) and  Support to Grid integration in LHC experiments (6-7 FTE’s)  As one of the LCG major contributors is developing the first World Wide Grid infrastructure for LHC experiments to be put in operation in June 2003  In coordination with the Tier1 set up team is building the INFN Grid infrastructure for LHC and HEP  Will set up a new WG with people of sites Tier3 candidates to define Tier3 M/W (extended eventually at international level)  Is now promoting EGEE to extend LCG infrastructure to other Sciences and Industries tests and to provide long term foundation for its support and operation  With the FIRB and IG-BIGEST is taking care of the integration of the Italian components  We believe that these are important outreach to guarantee in future current INFN level of funding in PNR

52. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-52 Responsabilities for FP6-BIGEST PROPOSAL  Mirco Mazzucato, Paolo Capiluppi, (Giorgio Maggi)......: Management  Antonia Ghiselli Coordinatore tecnico INFN per la preparazione del proposal  Francesco Prelz editor proposal tecnico INFN  Under finalization involvment of additional senior INFN computing managers in particular to set up eScience Centers

53. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-53 Conclusioni  Per fronte alle attivita’: LCG, FIRB IG-BIGEST e EGEE INFN Grid ha poposto una ridefinizione dell’ organizzazione del Progetto e delle responsabilita’.  L’ obiettivo, anche tenendo conto del quadro internazionale, e’ quello di avere una gestione efficace e unitaria di attivita’ “autonome”, che hanno raggiunto ciascuna un notevole livello di complessita’, mantenendo comunque una visione organica delle problematiche comuni.  E’ opinione dell’EB che solo mantenendo questa coerenza nella visione e nella gestione manageriale di tutte le attivita’ sovramenzionate si possano sviluppare sinergie sufficienti per permettere all’INFN di svilupparle in modo efficace con le risorse umane attualmente disponibili.

54. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-54 FIRB PROGETTI GRID: EDG, EDT, GLUE (EU-US) PROGETTO LHC COMPUTING GRID FP6 EU INFN GRID TIER1/2/3 INFN GARR CALCOLO ESPERIMENTI: LHC, VIRGO,APE, BABAR… Riassunto dei progetti e attivita’ a cui il progetto INFN Grid deve far fronte nella fase attuale.

55. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-55 Sotto-Progetti di INFN-Grid INFN-Grid Executive Board INFN-Grid Technical Board Collaborating grid projects DataGRIDDataTAG WP1 WP2 WP4 WP2 WP6 WP7 iVDGL GLUE FIRB WP3WP5 LCG FP6 Project Office BIGEST

56. last update 03/06/2015 12:29 Mirco Mazzucato INFN-Padova-56 The new INFN-GRID Management Structure Executive Board: Responsabile Progetto : Mirco Mazzucato Responsabili di attivita’: Paolo Capiluppi (FP6) Antonia Ghiselli (FIRB) Alberto Masoni (LCG) Laura Perini (EDT-EDG) Responsabile progettoTier-1: Federico Ruggieri Rappresentanti esperimenti: Paolo Capiluppi (CMS) Domenico Galli (LHCB) Massimo Masera (ALICE) Laura Perini (ATLAS), Raffaele Tripiccione (APE) Fulvio Ricci (VIRGO) Mauro Morandin BABAR Stefano Belforte CDF Rappresentante FIRB e Tech. Coordinator: Antonia Ghiselli Co-Tech. Coordinator per le applicazioni degli esperimenti: Lamberto Luminari Chair Institution Board: Lodovico Riccati Coordinatori Testbed INFN: Luciano Gaido e Lamberto Luminari Invitato in rappresentanza reti GARR e GEANT: Enzo Valente