Download presentation

Presentation is loading. Please wait.

Published byJackson Dalton Modified over 2 years ago

1
MLD2P4: a package of parallel algebraic multilevel Preconditioners Pasqua DAmbra, Institute for High-Performance Computing and Networking (ICAR-CNR), Naples Branch, Italy Bologna, March 2008 joint work with Daniela di Serafino, Second University of Naples Salvatore Filippone, University of Rome Tor-Vergata

2
Pasqua D'Ambra - Bologna March Overview Motivations Background Objectives MLD2P4: Multi-Level Domain Decomposition Parallel Preconditioners Package based on PSBLAS Algorithms and computational kernels Software architecture Some Results & Applications

3
Pasqua D'Ambra - Bologna March Background Large-scale applications have to solve The linear system matrix is: Real or complex and square Large and Sparse Distributed among parallel processors Matrix dimensions and entries, conditioning, sparsity pattern and coupling among variables vary along simulations

4
Pasqua D'Ambra - Bologna March Background (contd) What is the best method/preconditioner? No absolute winner, experimentation is needed Reliable preconditioners require access to the complete matrix Parallel implementation is not trivial Interfacing with application software is required Custom-made interfaces to parallel legacy codes Different interfaces for different preconditioners/solvers

5
Pasqua D'Ambra - Bologna March Objectives designing and implementing a suite of algebraic preconditioners based on Linear Algebra kernels for parallel sparse matrix computations Flexibility Different preconditioners by single API Portability & Efficiency Standard base software for serial kernels and data communications Simplicity of usage Modern (OO) Fortran 95 features and auxiliary routines for smooth legacy code integration

6
Pasqua D'Ambra - Bologna March MLD2P4 Multi-Level Domain Decomposition Parallel Preconditioners Package based on PSBLAS DiagonalBlock-Jacobi Additive Schwarz with arbitrary overlap Algebraic multi-level Schwarz PSBLAS Parallel Sparse Basic Linear Algebra Subprograms mld_prec_build(A,M,…) A, distributed sparse matrix (input) M, distributed sparse preconditioner (output) mld_prec_apply(M,x,y,…) M, distributed sparse preconditioner (input) x,y, distributed vectors (input/output)

7
Pasqua D'Ambra - Bologna March PSBLAS (Filippone et al., ) Basic Linear Algebra Operations with Sparse Matrices on MIMD Architectures Iterative Sparse Linear Solvers CG, BiCG, CGS, BiCGSTAB, RGMRES,… Appl. MPI BLACS Basic Linear Algebra Communication Subprograms F95 SBLAS (Duff et al.) Base sw Parallel Sparse Matrix Operations matrix-matrix products, matrix- vector products, … Kernels Parallel Sparse Matrix Management allocate, build, update, … F77

8
Pasqua D'Ambra - Bologna March MLD2P4 Design Algorithms Algebraic multi-level Schwarz preconditioners based on smoothed aggregation good trade-off between parallelism and convergence optimal scalability for symmetric positive-definite matrices algebraic framework allows general-purpose application

9
Pasqua D'Ambra - Bologna March (1-lev) Schwarz: basic ingredients Adjacency graph of A -overlap partition of W 0 -overlap partition of W

10
Pasqua D'Ambra - Bologna March AS: basic ingredients (contd) Restriction/prolongation operators Restriction of A

11
Pasqua D'Ambra - Bologna March Coarse level correction: basic ingredients Algebraic coarsening uncoupled aggregation Smoothed prol./restr. operators Coarse-level matrix

12
Pasqua D'Ambra - Bologna March Multilevel-Schwarz preconditioners & computational kernels Example: 2-lev hybrid-post build apply P. DAmbra, D. di Serafino, S. Filippone, On the Development of PSBLAS-based Parallel Two-level Schwarz Preconditioners, Applied Numerical Mathematics, 57, 2007.

13
Pasqua D'Ambra - Bologna March MLD2P4 Design Software Architecture Parallel Preconditioners BJA, ASM, RAS, ASH, ml-additive, ml-hybridpre, ml-hybridpost, ml-symmhybrid Appl. Preconditioner Build prolongation, restriction, coarse matrix, local sparse ILU and LU Kernels Preconditioner Application distributed & serial coarse matrix solvers PSBLAS 2.0 extended version of PSBLAS 1.0 Base sw

14
Pasqua D'Ambra - Bologna March Performance Results & Comparisons Different test matrices from various sources thm matrices: thermal diffusion in solids kivap matrices: automotive engine design shipsec matrices: from UF sparse matrix collection Experiments carried out on different Linux clusters 64 Intel Itanium dual-processor nodes connected by Quadrics QSNetII Elan 4 32 AMD Opteron dual-processor nodes connected by Myrinet 8 AMD Opteron dual-processor nodes connected by InfiniBand 8 Intel Itanium dual-processor nodes connected by Myrinet 16 Intel Pentium IV nodes connected by Fast Ethernet Comparison with up-to-date related work Trilinos-ML A. Buttari, P. DAmbra, D. di Serafino, S. Filippone, 2LEV-D2P4: a package of high-performance preconditioners for scientific and engineering applications, Applicable Algebra in Engineering, Communication and Computing, Vol. 18, 2007.

15
Pasqua D'Ambra - Bologna March Experimental Setting MLD2P4: right-preconditioned BiCGSTAB 1-lev Restricted Additive Schwarz preconditioner with ILU(0) (RAS) 2-lev hybrid Schwarz preconditioner, with RAS/ILU(0) as 1-lev prec. Distributed coarsest matrix: 4 sweeps of block Jacobi with ILU(0) (2LDI) or with UMFPACK (2LDU) on diagonal blocks 3-lev hybrid Schwarz preconditioner, with RAS/ILU(0) as 1-lev prec. Distributed coarsest matrix: 4 sweeps of block Jacobi with ILU(0) (3LDI) or with UMFPACK (3LDU) on diagonal blocks Stopping criterion: or maxit Unit right-hand side and null starting guess Row-block distribution of matrices: # submatrices = # procs

16
Pasqua D'Ambra - Bologna March thm matrices: number of iterations np OV=0 RAS2LDI2LDU3LDI3LDU thm1 n = nnz = Intel Itanium dual-processor nodes connected by QSNetII np OV=1 RAS2LDI2LDU3LDI3LDU

17
Pasqua D'Ambra - Bologna March thm matrices: execution times and speed-ups (OV=1; best execution times:3LDU) 64 Intel Itanium dual-processor nodes connected by QSNetII

18
Pasqua D'Ambra - Bologna March Application test case large eddy simulation of incompressible turbulent flows in a bi-periodical channel main computational kernel nonsymmetric and singular linear systems arising from elliptic PDE with Neumann b.c. A. Aprovitola, P. DAmbra, F. M. Denaro, D. di Serafino, S. Filippone, Application of Parallel Algebraic Multilevel Domain Decomposition Preconditioners in Large-Eddy Simulations of Wall-bounded Turbulent Flows: First Experiments, RT-ICAR-NA , July 2007.

19
Pasqua D'Ambra - Bologna March Experimental Setting MLD2P4: right-preconditioned RGMRES(30) 1-lev Restricted Additive Schwarz preconditioner with ILU(0) (RAS) 2-lev/3-lev hybrid Schwarz preconditioner, with RAS/ILU(0) as 1-lev prec. Distributed coarse matrix: 4 sweeps of block Jacobi with ILU(0) (2LDI/3LDI) on diagonal blocks Stopping criterion: or maxit General row-block distribution Pressure linear system n= nnz= Reynolds number: 180 Computational Grid: 140x32x45 non-uniform in the y direction, time-step 10 -4

20
Pasqua D'Ambra - Bologna March LES of incompressible wall-bounded flow 16 Intel Itanium dual-processor nodes connected by QSNetII SOR on 1 proc.=9 sec. SOR on 1 proc.=8580 sec.

21
Pasqua D'Ambra - Bologna March Work in progress Package available on the web very soon More sophisticated aggregation algorithms Integration of preconditioners and solvers in large-scale applications

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google