Presentation is loading. Please wait.

Presentation is loading. Please wait.

G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Portici 22 maggio 2013 The evolution of ENEAGRID/CRESCO HPC infrastructure G.Bracco

Similar presentations


Presentation on theme: "G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Portici 22 maggio 2013 The evolution of ENEAGRID/CRESCO HPC infrastructure G.Bracco"— Presentation transcript:

1 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Portici 22 maggio 2013 The evolution of ENEAGRID/CRESCO HPC infrastructure G.Bracco ENEA Centro Ricerche Frascati V. Enrico Fermi 45, Frascati (ROMA) S.Migliori,A.Quintiliani,S.Podda,R.Guadagni,F.Ambrosino,F.Beone, M.Caporicci,P.D'Angelo,A.Funel,G.Ponti,G.Furini,A.Mariano, G.Mencuccini,P.Ornelli,A.Perozziello,S.Pierattini,D.Abate,F.Poggi, D.Giammattei,M.DeRosa,S.Pecoraro,F.Simoni,S.Giusepponi, G.Guarnieri,A.Petricca,A.Rocchi,C.Sciò,A.Italiano, A.Colavincenzo,G.Giannini

2 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE The scientific computing resources of ENEA [The Italian National Agency for New Technologies,Energy and Sustainable Economic Development] are integrated in ENEAGRID infrastructure, a production quality, service oriented system for high performance and high throughput computing. The main computing resources are the CRESCO clusters (Linux x86_64), the main site is Portici (NA), the location of the original CRESCO project (2008, Computational RESearch center for COmplex systems), funded by the Italian Minister of Research and the Education,in the framework of PON This presentation describes the infrastructure and illustrates its evolution as funded by the new projects (PON framework) where ENEA-UTICT, the ICT Unit of ENEA, is one of the partners. Overview

3 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Outline ENEAGRID – The architecture & the computing resources – Users and applications – User interface and Virtual Labs – CRESCO clusters New projects PON – TEDAT, LAMRECOR New infrastructures – CRESCO3 – CRESCO4 Conclusions

4 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE The infrastructure is based on “mature” multiplatform software components assuring reliability and easy administration. Web interfaces have been developed/customized for a friendly user environment: – Kerberos 5 authentication – File systems: AFS/OpenAFS geographic file system (HOME) GPFS: parallel file system (also WAN) – Resource manager: LSF Multicluster – User Web graphical interface: NX/FARO Jobrama: job monitoring & Accounting – System monitoring: Zabbix – Web management of users/projects: WARC ENEAGRID architecture

5 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE ENEAGRID offers to the users computing resources based on Linux x86_64 ( ~5800 cores), AIX SP5 (~256 cpu), special systems (e.g. GPUs), virtualized hosts and distributed storage. The resources are located in 6 sites, connect by GARR network. ENEAGRID: computing resources ENE

6 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE ENEAGRID: CRESCO & Portici Site ENEA Research Center Portici Buildings designed by Vittorio Gregotti,1982/86

7 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE ENEAGRID users: – serial or small scale parallel jobs: ~200 users – large scale HPC parallel jobs: ~50 users Application domains: – combustion CFD – aerospace CFD – computational chemistry – climate modeling – atmospheric pollusion diffusion simulation – nuclear technologies – nuclear fusion physics – bioinformatics –... ENEAGRID: users and applications

8 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE ENEAGRID: applications Compilers: Intel, PGI, AMD Open64,... MPI Flavours:mvapich,openmpi,Intel,... Applications: Abaqus, Amira, Ansys, Ansys CFX, Ansys Fluent, AVS Express Developer Edition, COMSOL, CPMD, E-cell, Fluent, FreeMat, Gambit, Grass GIS, Gsharp, IcemCFD, IDL, LynxPrime, Marc, Mathematica, Matlab, MeshLab, ModeFrontier, MpCCI, MultiGen-Paradigm VegaPrime, Nastran, OpenFOAM, OpenSceneGraph, Paraview, Patran, Prince, Quantum Espresso, Quantum GIS, Scalasca, Scilab, Scirun, Siap, Starccm+, StarDesign, StarView, Tgrid, Totalview, VIsit, Visual Molecular Dynamics, Workbench, XILINX...

9 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO utilization by domain of activity Combustion CFD621.87ENEA New materials531.9ENEA, INFN/RM, INFN/NA,UniFI,UniSaUniSS, Numonyx Climate modeling357.57ENEA,Ylichron Nuclear fission122.9ENEA,ISS HPC Support activity103.11ENEA Atmospheric pollution65.62ENEA,AriaNet Aerospace CFD32.77Avio,AAPS,UniROMA1 University22.13UniROMA1,CERI Nuclear fusion19.99ENEA,PoliTo Bioinformatics18.3ENEA, CNR-ITB, Ylichron,CNR-ISA EFDA- Fusion4.69ENEA,EFDA Industry0.37NICE,CETMA

10 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 FARO - Fast Access to Remote Objects Web access to data, applications and virtual machines, a solution based on the integration of NX and a java interface

11 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 FARO & Virtual Labs Thematic portals integrating the access to data and to the applications for a specific context/activity

12 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Virtual Labs (1)

13 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Virtual Labs (2)

14 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Virtual Labs (3)

15 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Virtual Labs (4)

16 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Virtual Labs (5)

17 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Total displacement (mm ) Pos t Processo r Total displacement (mm ) Total displacement (mm ) Pos t Processo r Pos t Processo r FARO: 3D visualization 3D Remote Rendering

18 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 HPC: scalability test of various applications Fluent OpenFoam Commercial code Combustion Open Source: OpenFoam Processors User Code procs cpmd

19 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO clusters >Portici CRESCO1(672 cores 4U) CRESCO2(2720 cores blades HS21,GPFS)‏ IB CISCO 70xx, DDR Intel Clovertown,Tigerton,Nehalem,Westmere >Casaccia CRESCOC (192 cores, twin 1U,Supermicro)‏ IB Qlogic Silverstorm DDR, AMD 2427 Istanbul >Frascati CRESCOF (480 cores twin square 2U, GPFS)‏ IB Qlogic QDR, AMD 6172 Magnycours >Brindisi CRESCOB (80 cores, 4 U, GPFS)‏ GEthernet, Intel Tigerton >Trisaia CRESCOT (16 cores,4U) ‏ Intel Tigerton

20 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO1/2 Clusters - DDR IB & Storage SFS-7000D(D) SFS-7000D (e) SFS 7000D SFS P 4 graphic FRONT END 2X GPFS NODE SFS P 4X FRONT END FC (2) X 3850M2 10 X X HS21 SECTION 2 SM (ACTIVE) SM (standby X FRONT END GRAFICI 4X FRONT END 2X GPFS NODE 4) 4 4 2) 4 BACK-UP 4 IBM DCS TBDDN S2A TB 2X GPFS NODE IB 36 ports 4 20 TB Server STORAGE 20 TB Server STORAGE 20 TB Server STORAGE 20 TB Server STORAGE CRESCO2 Storage: IBM/DDN TB; DDN TB; disk servers 80 TB CRESCO1

21 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 New projects : PON PON/1 Ricerca Industriale DDR MIUR 01/Ric 18/1/2010 – - (application of new technologies to the conservation and valorization of cultural and artistic heritage) – LAMRECOR - (Advanced logistics for persons and goods) – DIRECTFOOD - Gestione integrata filiere alimentari e canali innovativi produttore -consumatore PON/2 Distretti/Laboratori Pubblico Privati DDR MIUR 713/Ric 29/10/2010 – VIS4FACTORY - Sistemi Informativi Visuali per i processi di fabbrica nel settore dei trasporti – DATABENC - Distretto ad ALta Tecnologia per i beni culturali nella regione Campania PON/3 Infrastrutture DDR MIUR 254/Ric 18/5/2011 – TEDAT - Centro di eccellenza per le Tecnologie e la diagnostica avanzata nel settore dei trasporti (Advanced technologies for transport,new materials for aerospace, automotive,..)

22 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 PON : Projects started 2012 Budget for infrastructures: [Ricerca industriale] ENEA-UTICT 320 k€ LAMRECOR [Ricerca industriale] ENEA-UTICT 548 k€ On these projects: new CRESCO3 cluster TEDAT [Infrastrutture] ENEA-UTICT k€ – Portici (2.056 k€) – Brindisi (165 k€) On this project: new CRESCO4 cluster

23 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Portici: new cluster CRESCO3 In the framework of and LAMRECOR projects the new CRESCO3 cluster has been set up in CRESCO computer room: 84 AMD nodes, 2016 cores, Twin Square SuperMicro & Acer, 19.3 TFlops peak Infiniband QDR QLogic (96 ports) 2 Racks (wide) The cluster is now opened for general use. HPL Benchmark: 75 % efficiency; compilers with Interlagos support (Open64 specific flags, Intel 12 specific flags; ACML Libraries) Storage: DDN S2A TB; GPFS

24 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE AMD Iterlagos GHz,64 GB,24 cores,84 nodes,2016 cores CRESCO3:SMicro/Acer twinsquare QDR IB

25 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Portici: CRESCO4 cluster In the framework of TEDAT project the procurament of the new CRESCO4 cluster is currently under way. An European tender has been set up, evaluated and the contract has been awarded (E4 Company). – 304 computing nodes Intel E2670, 2.6 GHz, 4864 cores – Supermicro Fat-Twin chassis (8 nodes in 4U) – 101 TFlops Peak – 5 Racks for computing nodes, 1 network rack – Storage: DDN S2A TB – Infiniband QDR QLogic/Intel (432 ports) – New computer room – Conditioning system taking advantage of free cooling technogy – Delivered in Portici by July 2013 – Final test completed October 2013

26 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE Portici: CRESCO4 - IB network

27 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE Computer rooms: CRESCO & CRESCO4 CRESCO computer room new CRESCO4 computer room

28 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO computer room (1)

29 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO computer room (2)

30 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO4 computer room (1)

31 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO4 computer room (2)

32 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO4 computer room (3)

33 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO4 computer room (4)

34 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE CRESCO4 computer room (5)

35 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 ENE Conclusions A significant update of the ENEAGRID HPC infrastructure is currently under way, with the new CRESCO3 and CRESCO4 clusters, funded by several PON projects The architecture of ENEAGRID provides the framework for an easy introduction of the new computing resources both from the point of view of the user and the administrator. By the end of the year the integrated peak computing power of ENEAGRID will increase of a factor 5 up to ~150 Tflops, the most powerfull system being CRESCO4 with ~100 Tflops.

36 G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Links


Download ppt "G. Bracco Evolution ENEAGRID/CRESCO - Portici 22/5/2013 Portici 22 maggio 2013 The evolution of ENEAGRID/CRESCO HPC infrastructure G.Bracco"

Similar presentations


Ads by Google