Large-scale accelerator simulations: Synergia on the Grid turn 1 turn 27 turn 19 turn 16 C++ Synergia Field solver (FFT, multigrid) Field solver (FFT, multigrid) single particle optics/utilities single particle optics/utilities wrapper/job control wrapper/job control glue input & lattice (MAD) input & lattice (MAD) analysis tools analysis tools results beam studies beam studies Python Fortran 90 C++ Octave, C++ software simulations data

Synergia ● Simulate multi-particle physics in accelerators ● Computationally intensive – 1-10's of millions of macro particles – 10's of thousands (or more) of PDE solves ● Massively parallel – Clusters and supercomputers ● 64-node Linux cluster typical ● 512 processors at NERSC C++ Synergia Field solver (FFT, multigrid) Field solver (FFT, multigrid) single particle optics/utilities single particle optics/utilities wrapper/job control wrapper/job control glue input & lattice (MAD) input & lattice (MAD) analysis tools analysis tools results beam studies beam studies Python Fortran 90 C++ Octave, C++

Running Synergia (1)Few-person collaboration (2)Simulations require complex input parameters (3)Output consists of many files (4)Need to take advantage of computing resources wherever they are available Grid computing is the answer for (4), but the increase in complexity arising from (2) and (3) has to be mitigated. The tools provided by Synergia allow the scientist to do science without getting bogged down by bookkeeping.

Computing on the Grid ● Scientist uses local resources for most tasks ● Remote systems used for computationally-intensive tasks only ● In our case, the computationally intensive tasks are running the simulations and some analysis job export job creation job DB analysis tools import results import results

Job creation ● Python-based system – Python not required on target site ● Job contains – Batch input ● created from template – Input files ● user-defined – Utilities ● clean output, pack output – Description ● human and machine readable Job directory batch file input files utility scripts description Goal is reproducibility

Managing job options ● Python module for command-line options ● Groups of options can be composed – General Synergia options – Batch options – Application-specific options – etc. ● Command-line is stored for cut-and-paste modification ● Automatic command-line help generation ● Automatic human- readable summary ● Options for created jobs can be added to database

Job database Job information is automatically entered in spreadsheet.

Results ● The measure of a scientific computing project is the science it produces ● The Synergia infrastructure has allowed us to produce more science with less time wasted on tedious tasks – Better utilization of resources – Less time spent bookkeeping – Fewer redundant simulation runs

The measure of a scientific computing project is the science it produces Fermilab Booster Accelerator