1. Simulation Production System Science Advisory Committee Meeting UW-Madison March 1 st -2 nd 2007 Juan Carlos Díaz Vélez

2. outline introduction production tools job management & error handling production sites & computing production challenges data storage and transfer

3. introduction SimProd written in Python interfaces easily with IceTray through Boost IceTray configurations written in well formatted XML are easy to store in database daemons manage cluster job submission SOAP interface for GUI client and for job monitoring

4. production tools GUI Production Client (& TUI) designed for IceTray configuration and job submission (simulation & offline processing)

5. production tools GUI Production Client (& TUI) designed for IceTray configuration and job submission (simulation & offline processing) Production Database store production history including all configured module parameters provide information on configurable parameters for client

6. production tools GUI Production Client (& TUI) Production Database store production history including all configured module parameters provide information on configurable parameters for client Production Server accepts dataset requests from client Provides job management including error handling Separate daemons handle dataset submission, queue/job management & monitoring

7. production tools GUI Production Client (& TUI) Production Database Production Server accepts dataset requests from client Provides job management including error handling Separate daemons handle dataset submission, queue/job management & monitoring Queuing Plugins(s) adaptable to different sites and batch systems

8. production tools GUI Production Client (& TUI) Production Database Production Server Queuing Plugins(s) adaptable to different sites and batch systems Logging/monitoring Database production status & troubleshooting remote job management unified monitoring for multiple clusters

9. production tools GUI Production Client (& TUI) Production Database Production Server Queuing Plugins(s) Logging/monitoring Database production status & troubleshooting remote job management unified monitoring for multiple clusters Web Interface (Ian Rae) cluster/dataset/job monitoring search engine for production db dataset statistics

10. job management job goes through series of states Web Interface (Ian Rae) job/server communication job eviction file transfer error

11. production sites current simulation production grid

12. computing quite significant computing in the collaboration different architectures and OS different batch systems different policies each site provides a contact person as local responsible contact person keep in touch with sys admin maintain production monitor runtime & completion troubleshooting system be sure data are stored data integrity

13. photonics photon interaction probability tables are produces with detailed module of ice properties full set of tables is ~14 GB (too large for memory (32-bit) we sort events in zenith bins and process process each bin separately. current production clusters have tables pre-installed on nodes This limits our ability to add new clusters or large grids for simulation production.

14. data storage archive documented through DIF metadata DIF (Directory Interchange Format) adapted to Astrophysics SimProd automatically generates DIF from simulation parmeters. pending: interface simprod with datawarehouse Ingest system files collected from sites and stored at UW local responsibles manually transfer data to UW pending: automatic data movement from sites (testing gridftp)

15. work in progress some features planned for SimProd automatic file transfer (GridFTP) dynamic collection and reporting of detailed simulation statistics. better user interface. Search engine will allow user to display datasets based on different criteria (e.g. geometry, primary spectrum, grid site, etc.)

16. links Simulation Production Web Page http://internal.icecube.wisc.edu/simulation sim-prod documentation and wiki http://wiki.icecube.wisc.edu/index.php/SimProd