Progress on Component-Based Subsurface Simulation I: Smooth Particle Hydrodynamics Bruce Palmer Pacific Northwest National Laboratory Richland, WA

Smooth Particle Hydrodynamics Use a discrete sampling of points to approximate continuum hydrodynamic fields where W is a smooth, normalized weighting function with support h and ρ is

Smooth Particle Hydrodynamics The equations of motion for the particles are where P is the pressure and Π is a stress tensor that depends only on the properties of particles at locations i and j.

Smooth Particle Hydrodynamics Algorithmically similar to molecular dynamics Explicit time integration of particle trajectories Finite range, pairwise interactions between particles For large simulations, particles are distributed on processors based on spatial location

Spatial Distribution P0P0 P1P1 P2P2 P3P3 P4P4 P5P5 P6P6 P7P7 Spatial Decomposition Algorithm: Partition particles among processors Update coordinates at every step Update partitioning after fixed number of steps

Componentization Defining components: What functionality and data belong together in a component Granularity: At what level is componentization compatible with performance? Abstraction of Interfaces: Can interfaces be defined that support multiple implementations representing different models and/or algorithms? Resource Allocation: Which components allocate memory and how is this communicated to other components?

CCA Framework for SPH Three major components (and numerous minor ones) Time integration component: update coordinates and velocities at each time step Force evaluation component: Evaluate forces on each particle Communication component: manage movement of particle data between processors

CCA Framework for SPH Communication Forces Time Integration

It works! (Mostly) SPH simulation of flow in porous media on MPP2

Communication layer Use integer handles to label each “action” specified by user 5 types of action supported: “shuffle”, “sort”, “update”, “gather”, “scatter-add” Actions support the distribution of particles to processors based on spatial location and the gathering and scattering of particles into and from a buffer region of some user-specified width surrounding the spatial domain owned by individual processors

Communication layer Each handle can be assigned an action. An arbitrary number of integer or double vectors pairs can also be assigned to this handle. When the user calls the transfer function on a handle, all data associated with that handle is moved according to the appropriate action

Communication layer The communication layer is also the repository of geometry information (simulation size, cell size) Only coordinates and particle index have any special status

Observations, Requests, etc. Developer environment (we need one) –Where’s Bocca? –Support for multiple files in components –Support top level setting of libraries, compiler options, etc. These can be overwritten at component level –Some kind of facility for duplicating existing components to new components (facilitates experimentation) –Can a component be easily generated if a Babel interface for a software package already exists? (ITAPS)

Observations, Requests, etc. 32 and 64 bit platforms –Integer arrays are particularly a problem (I think kluges exist for single variables) –sidl_integer data type?

Observations, Requests, etc. Interactions with workflow environments (I think some of this stuff already exists but is undocumented) –Command line arguments –Support for versioning documentation that can be exported to output files –Need some way of extracting port names without looking at setServices method (currently the only way to do this is to use the GUI)

Observations, Requests, etc. Documentation –How do you use sidl_opaque data types? –How do exceptions work?