1. Storage Management on the Grid Alasdair Earl University of Edinburgh

2. Talk outline Motivation Storage management issues Existing projects Current work Conclusion

3. CERN capacity requirements

4. Cost Initially record ~4PB / year 1 PB = 1 Tape silo 1 silo = £500k Media = £600k + overhead … Tape is the cheapest option but has issues Grid is the solution LHC computing expensive

5. LHC computing challenge Tier2 Centre ~1000 PCs Online System Offline Farm ~20,000 PCs CERN Computer Centre >20,000 PCs RAL Regional Centre US Regional Centre French Regional Centre Italian Regional Centre Institute Institute ~200 PCs Workstations ~100 Mbyte/sec ~100 MByte/sec 100 - 1000 Mbit/sec one bunch crossing per 25 ns 100 triggers per second each event is ~1 Mbyte physicists work on analysis “channels” each institute has ~10 physicists working on one or more channels data for these channels is cached by the institute server Physics data cache ~PByte/sec ~ Gbit/sec or Air Freight Tier2 Centre ~1000 PCs ~Gbit/sec Tier 0 Tier 1 Tier 3 Tier 4 assumes PC = ~ 25 SpecInt95 ScotGRID++ ~1000 PCs Tier 2

6. Competing requirements A durable, long term store for large amounts of data –Very long term data (centuries) –Very reliable storage (as good as paper) Metadata –Overview / administration information –Individual file information A security mechanism for access and modification Online access to data

7. Requesting data from a storage system user index Data storage system(s) 1 2 3 4 5 Staging area Request manager

8. Storage Element Storage for European DataGrid Developed primarily by RAL under EDG Work Package 5 Metadata in XML –aiming for OGSA compliance Used by physics projects WP5 Logo

9. Storage Element Protocols: FTP, HTTP(s), GridFTP, scp … MSS: Mass Storage System - disk, tape, hierarchy … Old ModelNew Model

10. Storage Resource Broker Part of the NSF funded Data Intensive Computing Environments (DICE) project San Diego Supercomputing Center Centralised metadata system –Dublin Core In use with CMS and BaBar Large following in BioInformatics apps

11. Storage Resource Broker SRB MCAT DB SRB 1 2 3

12. Storage Resource Manager Aim: High performance data storage –Teragrid –Supercomputing apps (high data rate) Implemented by LBNL and FNAL 2 standards available –Process based –Web Services

13. Storage Resource Manager 1 2 4 5 3

14. Current work Grid storage is currently specialised –Each project has advantages and disadvantages Need for interoperation between existing projects UK interest –Interoperation of existing resources –National level –Local level

15. Current Work ScotGrid –Durham –Edinburgh –Glasgow Rollout of storage management software Discussions for joint project Strong push to solve interoperation issues Scottish Tier 2 Centre

16. Conclusions We have presented 3 major projects Storage management on the Grid is rapidly advancing The existing project teams are working together Edinburgh trying to solve some of the interoperation issues