1. Parallel Unstructured Mesh Infrastructure
FASTMath Team Members: Dan Ibanez, E. Seegyoung Seol, Gerrett Diamond, Cameron W. Smith, Qiukai Lu, Mark S. Shephard Scientific Computation Research Center, Rensselaer Polytechnic Institute
The PUMI toolkit supports massively parallel adaptive unstructured mesh applications. It also supports in-memory integration of adaptive mesh control through APIs. Current in-memory integration with PHASTA, Proteus, Nektar++, and FUN3D for fluid flow analysis, ACE3P for electromagnetics analysis, M3D-C1 for MHD analysis of fusion plasmas, and Albany for multiphysics analysis.
PUMI
Component-based Design
***
****
PPPL Fusion Codes
PUMI consists of 40K lines of C/C++ code separated into interchangeable components.
PUMI is a set of C/C++ libraries for representing and manipulating parallel unstructured meshes for large-scale finite element simulations. Key components include:
PCU – a parallel communication library built on MPI, supporting sparse data exchange
MDS – a compact mesh data structure, combining modifiability and array storage
APF – a common mesh and field API enabling dependency inversion
GMI – a common geometric model API, also serving dependency inversion of CAD kernels
ParMA – a set of highly scalable partitioning tools
In collaboration with the Princeton Plasma Physics Lab, PUMI is being used to represent fusion reactor meshes. The mesh may be highly constrained, with several concentric rings specified by analysis of the electromagnetic behavior.
Associated tools:
SPR – error estimator
MeshAdapt – mesh adaptation based on PUMI components
Chef – preprocessor for PHASTA workflow .
Partitioning
***
****
***
****
Curved Meshes for High-order Methods
PUMI includes scalable mesh partitioners in the ParMA component, which have been applied to billion element meshes on hundreds of thousands of cores.
Complex curved geometry can be better approximated by high-order shape functions for surface faces. With proper corrections to maintain a positive Jacobian, this particle accelerator can be meshed with very coarse 4th-order faces.
PUMI integrates with other scalable unstructured mesh libraries. Meshes and models from the libraries of Simmetrix, Inc. can be brought into PUMI through the APF abstraction. PUMI meshes and fields can be copied to the Trilinos STK package from Sandia National Labs.
***
****
Recent work includes element weights for mixed meshes that balance memory use.
BL elements can otherwise overflow on-node memory.
***
****
Modifiable Array Structure
PUMI has an array storage implementation (MDS) which efficiently supports entity-level modification, while being more memory efficient than other implementations capable of general adaptation.
Fields
Scaling
Linked lists in arrays allow O(1)
Updates to upward adjacency structures while using same memory as static versions.
PUMI implements fields with support for local coordinates and coordinate transformation.
apf::Field supports various polynomial and tensor orders.
MeshAdapt transfers fields automatically with proper interpolation.
Meshes up to 92B elements on 256K processes on ALCF’s Mira using hybrid parallelism of 8 threads per MPI rank
More Information: or contact Dan Ibanez, RPI,