Particle Transport Methods in Parallel Environments   Prof. A. Haghighat (haghighat@psu.edu) Nuclear & Radiological Engineering Department University of Florida 202 Nuclear Sciences Building Gainesville, FL 32611, USA [Prepared for a meeting with Dell/HPC visitors, Dec. 13, 2002]

Objective Simulation of particle transport of real-life nuclear systems (power reactors, medical devices, detection devices) for design and optimization

Particle Transport Theory Expected number of particles in a phase space (dVdEd) at time t: n(r, E, Ω, t) dVdEd z Ω dE dΩ dv r y x Definitions Angular flux - Scalar flux - Reaction rate density = macroscopic cross-section

Particle Transport Theory Approaches Deterministic Statistical Monte Carlo

Deterministic Transport Theory Approach Linear Boltzmann equation (steady-state) Balance equation for expected number of particles in a phase space (dVdEd); Streaming Collision Scattering Independent source Fission

Sn Method The Sn balance equation for a spatial grid, group g, direction m is given by where There are 6 independent variables, e.g., x,y,z, E,  and 

Issues Need for large computer memory, e.g., for a typical shielding problem with 100x100x100 (space) x 80 (directions)x 50 (groups) memory required = 32 GB ! Need for significant computation time because of slow convergence

PENTRANTM (Parallel Environment Neutral-particle TRANsport) and its application to Real-Life Nuclear Systems 8 UFTTG

PENTRANTM (1) Parallel Environment Neutral-particle TRANsport developed from scratch in 1996: ANSI FORTRAN F77/f90 with MPI library, over 33,000 lines Industry standard FIDO input Solves 3-D Cartesian, multigroup, anisotropic transport problems Forward and adjoint mode Fixed source, criticality eigenvalue problems Parallel processing algorithms Full phase-space decomposition: Parallel in angle, energy, and spatial variables 9 UFTTG

PENTRANTM (2) (continued) Numerical formulations Parallel I/O Partitioned memory for memory intensive arrays (angular fluxes, etc) Builds MPI processor communicators Automatic scheduling using a decomposition weighting vector Numerical formulations Adaptive Differencing Strategy Diamond Zero (DZ) Directional Theta-Weighted differencing (DTW) Exponential-Directional Weighted (EDW) 10 UFTTG

PENTRANTM (4) (continue) Allows for a fully discontinuous variable meshing between coarse meshes: Uses a novel higher order mesh coupling scheme: Taylor Projection Mesh Coupling (TPMC) Acceleration Spatial Two-grid (“/”) with TPMC Angular multigrid (AMG) formulations with TPMC PCR with a zoned rebalance acceleration AMG + PCR Iterative techniques Multigroup & One-level SI schemes 11 UFTTG

PENTRANTM (5) Red-Black and Block Jacobi iteration Anisotropic scattering via Legendre moments through P7, Angular quadrature set: Level symmetric (up to S20) with ordinate splitting (OS) Pn-Tn with OS Vacuum, reflective, and albedo boundaries Volumetric & planar angular sources 12 UFTTG

PENTRAN Performance Achieved high parallel fractions of 90-98% for solving real-life problems such as a BWR core shroud; a PWR cavity dosimetry; CT scan; X-ray room Compared well with theoretical predictions

Monte Carlo Methods Simulation of a physical process on a computer by sampling PDFs of basic physical processes using Random Numbers Issues Large computation time Achieving small variance Solution Parallel Monte Carlo (it is straightforward; embarrassingly parallel) Variance reduction methods (developed automated adjoint-based algorithms)

Activities/interests Parallel computing PTDC laboratory at NRE contains 2 PC clusters: 8 machines with 2 GB memory dedicated to LBE 6 machines with 1 GB memory dedicated to Monte Carlo Web-based Computing PENTRAN code system is available on the web High Performance Computing graduate minor/certificate We are preparing a proposal