UKQCD QCDgrid Richard Kenway

UKQCD Nov 2001QCDgrid2 why build a QCD grid? the computational problem is too big for current computers –configuration generation is tightly coupled  a few teraflops machines –post-processing is highly diverse and distributed it involves multinational collaborations –virtual organisations are well established and growing many terabytes of data should be federated –validity + security are essential, so data provenance is a vital issue extensive software exists and should be more widely exploited –must be correct, portable and efficient QCD configurations

UKQCD Nov 2001QCDgrid3 APEmille Swansea QCDgrid infrastructure QCDOC Edinburgh QCDOC BNL QCDOC Columbia TB disk farm Glasgow TB disk farm Edinburgh TB disk farm Liverpool TB disk farm Swansea US Tier II centers German/Italian Tier II centres Tier I Tier II Tier III apeNEXT DESY/INFN ScotGrid Ulgrid

UKQCD Nov 2001QCDgrid4 data mark up 1label configurations by physics parameters and history data in XML format extensible schema propose to SciDAC as US standard translation programs: UKQCD  Columbia  US  … UKQCD data catalogue  logical file name  EDG replica management system  physical file name milestone Feb 02: UKQCD data catalogue implemented on top of EDG TB1 D1 primary data: L, , {c}, {  sea }, {history} several  1000  1 GB D2 secondary data: L, , {c}, {  sea },  valence, {history} several  1000  5  5 GB D3 tertiary data: L, , {c}, {  sea }, {  valence }, {f}, {history} many  20 MB total UKQCD data = hundred TB per year progress: XML schema working party to report in Dec physics history software

UKQCD Nov 2001QCDgrid5 data storage and distribution 2federate (global) data with open + restricted access data generation at Edinburgh, Columbia and Swansea data replication across TB RAID disk farms at Edinburgh, Glasgow, Liverpool and Swansea to be fully operational well before our existing data store closes in Sep 02 possibly extending to the rest of the US, Germany and Italy milestone Jun 02: functioning distributed data store 1. users store data according to physics parameters and update catalogue 2. replica management system generates a geographically separate copy 3. users fetch data by physics parameters 4. replica management system optimises data retrieval progress: RAID arrays installed at Edinburgh, Glasgow & Liverpool EDG TB1 tests beginning this week (Certification Authority?)

UKQCD Nov 2001QCDgrid6 user interface 3provide a high-level interface to data, codes and machines web portal with increasing grid functionality to automate and distribute data analysis jobs to be operational for QCDOC data analysis beginning in 2003 milestone Jun 02: portal supports data management Oct 02: portal supports local job submission via scripts Jun-Oct 03: portal supports distributed data analysis progress: use cases specified (important input to schema)

UKQCD Nov 2001QCDgrid7 resources required 2 FTE of programmer effort over the next 24 months –assumes appropriate EDG middleware is available via GridPP data catalogue and user interface require dedicated effort (1 FTE) –managed by Alan Irving grid implementation shared with GridPP (1 FTE) –how will UKQCD project management draw on (distributed) GridPP resources? QCDgrid is a relatively simple grid application which implements generic grid functionality and must support real users by the end of 2003