1. DESC mtg U Penn 11-13 June, 2012 Computing Infrastructure Computing Parallel Session R.Dubois richard@slac.stanford.edu

2. DESC mtg U Penn 11-13 June, 2012 Potential Work Areas Management of simulations –Sources of cycles –Manpower to run them workflow engine to minimize effort? –Managing the data that results: catalogue and access Look at usability of tools –Connection between cosmo sims and ImSim –Harnesses, config tools for running ImSim –Long term support for ImSim? Collaborative tools: foster communications and sharing –Forum, wiki Computing Parallel Session

3. DESC mtg U Penn 11-13 June, 2012 Managing Simulations Clusters at collaboration sites Grid cycles John asking for 3 FTEs to run sims Data catalogue to track simulated data –Probably no off-the-shelf tool currently to track data across arbitrary sites and provide access tools –Examples from Fermi: Catalogue – being adapted to LSST camera test data as demo Workflow engine – web based submission/tracking tool –Central submission to multiple sites with addition of simple interface to local batch system –Being used at SLAC for ImSim runs now as demo Computing Parallel Session

4. DESC mtg U Penn 11-13 June, 2012 Catalogue Example Computing Parallel Session

5. DESC mtg U Penn 11-13 June, 2012 Example Work Flow Engine Computing Parallel Session

6. DESC mtg U Penn 11-13 June, 2012 Running ImSim Computing Parallel Session User Friendliness: 10 CPU-hrs per visit Several steps Understand DM tools: Best path to catalogues

7. DESC mtg U Penn 11-13 June, 2012 ImSim Usability Aids Have a recipe for running – needs user intervention between steps –Can we streamline the recipe? Might the code benefit from profiling? May be some low hanging fruit to make it go faster than 10 hrs per visit? Might not be hard to keep a repository of compiled versions of ImSim for popular OSes –Fetch everything needed via an installer If we hope for a broader developer community –Maybe look at packaging and internals to enable/encourage new developers to get involved Computing Parallel Session

8. DESC mtg U Penn 11-13 June, 2012 Collaborative Tools Common repository for collaboration code –Can create a mix of “user contributions” which can evolve into a collection of integrated tools in the same repository Ease of collaboration: –Use a forum for discussion More powerful than mailing lists Use good wiki tool for sharing and discussing documents, analyses etc Computing Parallel Session

9. DESC mtg U Penn 11-13 June, 2012 Some Open Questions Can there be a common framework defined for analysis tools? At what level? FITS atomic model with files as interfaces? What does LSST project framework look like? Can/should it be emulated? Should there be common tools defined for things like plotting etc? Any connection to other surveys? How to share/store other surveys' data? What about manpower for supporting all this? Analysis tasks & systematics session Verification & Validation of project pipelines Plea for cross experiment computing resources What is 'deliverable' for systematics studies? Could there be data flow diagrams to help identify any? Non-ImSim sims?? –Need fast sim capability Computing Parallel Session