1. www.egi.eu EGI-InSPIRE RI-261323 EGI-InSPIRE www.egi.eu EGI-InSPIRE RI-261323 Shared Services and Tools MPI John Walsh, Isabel Campos, Antonio Laganà EGITF-2010, Amsterdam

2. www.egi.eu EGI-InSPIRE RI-261323 MPI in EGI MPI support in ARC, gLite, UNICORE JS mechanisms in M/Ws differ radically –ARC: xRSL + RTE –gLite: JDL with MPI-START wrappers –UNICORE: JSDL HUCs will exploit MPI enabled resources made available through these middlewares

3. www.egi.eu EGI-InSPIRE RI-261323 MPI Heavy Users CompChem & Fusion –Very early drivers of MPI on EGEE But also other major scientific discipline! –Astronomy & Astrophysics –Biomed and Life Sciences –Earth Sciences –……

4. www.egi.eu EGI-InSPIRE RI-261323 MPI in EGI-InSPIRE Heavy User Community focus –Task TSA3.2.5 (Support for HUC) –54 PM over three years (CSIC, TCD, UNIPG) SA1 Early Adopters SA1 effort from NGIs to deal with acc/ issue MPI related components from EMI –MPI-START and glite-MPI_utils, YAIM MPI –Alternative/Custom MPI packaging (optional) Support better batch system integration

5. www.egi.eu EGI-InSPIRE RI-261323 People Isabel Campos (CSIC) Antonio Laganà (UNIPG) Alessandro Costantini (UNIPG) * Enol Fernández del Castillo (CSIC) Stefano Crocchianti (UNIPG) Osvaldo Gervasi (UNIPG) Carlo Manuali (UNIPG) Leonardo Pacifici (UNIPG) * Jamie Shiers (CERN) John Walsh (TCD)

6. www.egi.eu EGI-InSPIRE RI-261323 MPI HUC objectives Improved end-user documentation –Deployment –job submission in ARC, gLite and UNICORE –MPI application development Outreach and dissemination User community, NGI, site engagement –Gathering direct input and requirements Standardisation bodies participation Generic Parallel Job support (GPU/OpenMP)

7. www.egi.eu EGI-InSPIRE RI-261323 MPI Task Outputs MPI Workbenches –Increasing complexity Simple -> complex workflows –Focus on HUC communities –Reuse by other communities User and Site Admin surveys –To specify future m/w requirements Dissemination MPI Cookbook

8. www.egi.eu EGI-InSPIRE RI-261323 Parallel Applications (CCMST) Basic Linear algebra algorithms (low level complexity codes) –Cannon, Fox & Strassen algorithms Medium complexity (in house codes) –RWAVEPR quantum time-dependent reactive dynamics –SC-IVR semiclassical -initial value representation- High complexity (general purpose computational codes) –GAMESS-US high-level ab initio molecular quantum chemistry –DL_POLY molecular dynamics simulation of complex systems –NAMD molecular dynamics simulation of complex systems

9. www.egi.eu EGI-InSPIRE RI-261323 Large variety of Applications: Serial, parallel, shared memory,… BUT: Outstanding Parallel long simualtions (MPI on HPC). GYROKINETIC SIMUATION OF PLASMA TURBULENCE Example: EUTERPE code (30 days CPU for 300 micro-second evolution on Mare Nostrum at BSC, Spain) Target: 10 seconds of evolution. MPI in Fusion

10. www.egi.eu EGI-InSPIRE RI-261323 Workflows involving Grid-HPCs, serial and parallel applications. Kepler launches the different actors and organizes the workflow. Kepler runs on the fusion Gateway for managing the data. One actor running on HPC: MPI. And the other on the grid (thousands of jobs): Serial or MPI. HPC Code (MPI) Grid Code (serial or MPI) Grid Code (serial or MPI) Fusion MPI Workflows Fusion slides courtesy of CIEMAT: Francisco M. Castejon Magaña, Antonio Gomez Iglesias. Fusion VO KEPLER: Workflow engine Gateway KEPLER: Workflow engine Gateway

11. www.egi.eu EGI-InSPIRE RI-261323 EGI Virtuous Cycle UsersRequirementsTechnologyAssessmentInfrastructure

12. www.egi.eu EGI-InSPIRE RI-261323 Issues Infrastructure in transition –Integration and interoperability path Nagios or equivalent for MPI testing? Standards for MPI/Parallel apps/libs? MPI from some OS distros not suitable Accounting –Torque deficiencies, potentially others –GPU usage accounting? –Multi-core/Multi-thread?

13. www.egi.eu EGI-InSPIRE RI-261323 Questions?