EGEE is a project funded by the European Union under contract IST HEP Use Cases for Grid Computing J. A. Templon Undecided (NIKHEF) Grid Tutorial, NIKHEF Amsterdam, 3-4 June

NIKHEF Grid Tutorial, 3-4 June Contents The HEP Computing Problem How it matches the Grid Computing Idea Some HEP “Use Cases” & Approaches

NIKHEF Grid Tutorial, 3-4 June Our Problem  Place event info on 3D map  Trace trajectories through hits  Assign type to each track  Find particles you want  Needle in a haystack!  This is “relatively easy” case

NIKHEF Grid Tutorial, 3-4 June More complex example

NIKHEF Grid Tutorial, 3-4 June interactive physics analysis batch physics analysis batch physics analysis detector event summary data raw data event reprocessing event reprocessing event simulation event simulation analysis objects (extracted by physics topic) event filter (selection & reconstruction) event filter (selection & reconstruction) processed data Data Handling and Computation for Physics Analysis

NIKHEF Grid Tutorial, 3-4 June Scales To reconstruct and analyze 1 event takes about 90 seconds Maybe only a few out of a million are interesting. But we have to check them all! Analysis program needs lots of calibration; determined from inspecting results of first pass.  Each event will be analyzed several times!

NIKHEF Grid Tutorial, 3-4 June online system multi-level trigger filter out background reduce data volume level 1 - special hardware 40 MHz (40 TB/sec) level 2 - embedded processors level 3 - PCs 75 KHz (75 GB/sec) 5 KHz (5 GB/sec) 100 Hz (100 MB/sec) data recording & offline analysis One of the four LHC detectors

NIKHEF Grid Tutorial, 3-4 June Scales (2) 90 seconds per event to reconstruct and analyze 100 incoming events per second To keep up, need either:  A computer that is nine thousand times faster, or  nine thousand computers working together Moore’s Law: wait 20 years and computers will be 9000 times faster (we need them in 2007!)

NIKHEF Grid Tutorial, 3-4 June Computational {Impli,Compli}cations Four LHC experiments – roughly 36k CPUs needed BUT: accelerator not always “on” – need fewer BUT: multiple passes per event – need more! BUT: haven’t accounted for Monte Carlo production – more!! AND: haven’t addressed the needs of “physics users” at all!

NIKHEF Grid Tutorial, 3-4 June LHC User Distribution

NIKHEF Grid Tutorial, 3-4 June Classic Motivation for Grids Trivially parallel problem Large Scales: 100k CPUs, petabytes of data  (if we’re only talking ten machines, who cares?) Large Dynamic Range: bursty usage patterns  Why buy 25k CPUs if 60% of the time you only need 900 CPUs? Multiple user groups (& purposes) on single system  Can’t “hard-wire” the system for your purposes Wide-area access requirements  Users not in same lab or even continent

NIKHEF Grid Tutorial, 3-4 June Solution using Grids Trivially parallel: break up problem “appropriate”-sized pieces Large Scales: 100k CPUs, petabytes of data  Assemble 100k+ CPUs and petabytes of mass storage  Don’t need to be in the same place! Large Dynamic Range: bursty usage patterns  When you need less than you have, others use excess capacity  When you need more, use others’ excess capacities Multiple user groups on single system  “Generic” grid software services (think web server here) Wide-area access requirements  Public Key Infrastructure for authentication & authorization

NIKHEF Grid Tutorial, 3-4 June HEP Use Cases Simulation Data (Re)Processing Physics Analysis General ideas presented here … contact us for detailed info

NIKHEF Grid Tutorial, 3-4 June Simulation The easiest use case  No input data  Output can be to a central location  Bookkeeping not really a problem (lost jobs OK) Define program version and parameters Tune # of events produced per run to “reasonable” value Submit (needed ev)/(ev per job) jobs Wait

NIKHEF Grid Tutorial, 3-4 June Data (Re)Processing Quite a bit more challenging: there are input files, and you can’t lose jobs One job per input file (so far) Data distribution strategy Monitoring and bookkeeping Software distribution Traceability of output (“provenance”)

NIKHEF Grid Tutorial, 3-4 June km3net Reconstruction Model Mediterranea n 10 Gb/s L1 Trigger Raw Data Cache > 1 TB > 1000 CPUs  Distributed Event Database?  Auto Distributed Files?  Single Mass Store + “Thermal Grid”? StreamService 1 Mb/s This needs work!! 2 Gbit/s is not a problem but you want many x 80 Gbit/s! Dual 1TB Circular Buffers? Distribute from shore station? Or dedicated line to better- connected location, distribute from there?? Grid useful here – get a lot but only when you need it! Grid data model applicable, but maybe not computational model …

NIKHEF Grid Tutorial, 3-4 June HEP Analysis Model Idea

NIKHEF Grid Tutorial, 3-4 June Conclusions HEP Computing well-suited to Grids HEP is using Grids now There is a lot of (fun) work to do!