EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Shared Services and Tools MPI John Walsh, Isabel Campos, Antonio Laganà EGITF-2010, Amsterdam

EGI-InSPIRE RI MPI in EGI MPI support in ARC, gLite, UNICORE JS mechanisms in M/Ws differ radically –ARC: xRSL + RTE –gLite: JDL with MPI-START wrappers –UNICORE: JSDL HUCs will exploit MPI enabled resources made available through these middlewares

EGI-InSPIRE RI MPI Heavy Users CompChem & Fusion –Very early drivers of MPI on EGEE But also other major scientific discipline! –Astronomy & Astrophysics –Biomed and Life Sciences –Earth Sciences –……

EGI-InSPIRE RI MPI in EGI-InSPIRE Heavy User Community focus –Task TSA3.2.5 (Support for HUC) –54 PM over three years (CSIC, TCD, UNIPG) SA1 Early Adopters SA1 effort from NGIs to deal with acc/ issue MPI related components from EMI –MPI-START and glite-MPI_utils, YAIM MPI –Alternative/Custom MPI packaging (optional) Support better batch system integration

EGI-InSPIRE RI People Isabel Campos (CSIC) Antonio Laganà (UNIPG) Alessandro Costantini (UNIPG) * Enol Fernández del Castillo (CSIC) Stefano Crocchianti (UNIPG) Osvaldo Gervasi (UNIPG) Carlo Manuali (UNIPG) Leonardo Pacifici (UNIPG) * Jamie Shiers (CERN) John Walsh (TCD)

EGI-InSPIRE RI MPI HUC objectives Improved end-user documentation –Deployment –job submission in ARC, gLite and UNICORE –MPI application development Outreach and dissemination User community, NGI, site engagement –Gathering direct input and requirements Standardisation bodies participation Generic Parallel Job support (GPU/OpenMP)

EGI-InSPIRE RI MPI Task Outputs MPI Workbenches –Increasing complexity Simple -> complex workflows –Focus on HUC communities –Reuse by other communities User and Site Admin surveys –To specify future m/w requirements Dissemination MPI Cookbook

EGI-InSPIRE RI Parallel Applications (CCMST) Basic Linear algebra algorithms (low level complexity codes) –Cannon, Fox & Strassen algorithms Medium complexity (in house codes) –RWAVEPR quantum time-dependent reactive dynamics –SC-IVR semiclassical -initial value representation- High complexity (general purpose computational codes) –GAMESS-US high-level ab initio molecular quantum chemistry –DL_POLY molecular dynamics simulation of complex systems –NAMD molecular dynamics simulation of complex systems

EGI-InSPIRE RI Large variety of Applications: Serial, parallel, shared memory,… BUT: Outstanding Parallel long simualtions (MPI on HPC). GYROKINETIC SIMUATION OF PLASMA TURBULENCE Example: EUTERPE code (30 days CPU for 300 micro-second evolution on Mare Nostrum at BSC, Spain) Target: 10 seconds of evolution. MPI in Fusion

EGI-InSPIRE RI Workflows involving Grid-HPCs, serial and parallel applications. Kepler launches the different actors and organizes the workflow. Kepler runs on the fusion Gateway for managing the data. One actor running on HPC: MPI. And the other on the grid (thousands of jobs): Serial or MPI. HPC Code (MPI) Grid Code (serial or MPI) Grid Code (serial or MPI) Fusion MPI Workflows Fusion slides courtesy of CIEMAT: Francisco M. Castejon Magaña, Antonio Gomez Iglesias. Fusion VO KEPLER: Workflow engine Gateway KEPLER: Workflow engine Gateway

EGI-InSPIRE RI EGI Virtuous Cycle UsersRequirementsTechnologyAssessmentInfrastructure

EGI-InSPIRE RI Issues Infrastructure in transition –Integration and interoperability path Nagios or equivalent for MPI testing? Standards for MPI/Parallel apps/libs? MPI from some OS distros not suitable Accounting –Torque deficiencies, potentially others –GPU usage accounting? –Multi-core/Multi-thread?

EGI-InSPIRE RI Questions?