1. Jean-Yves Nief, CC-IN2P3 Wilko Kroeger, SCCS/SLAC Adil Hasan, CCLRC/RAL HEPiX, SLAC October 11th – 13th, 2005 BaBar data distribution using the Storage Resource Broker (SRB)

2. HEPiX Conference, SLAC, October 11th-13th 2005 2 BaBar overview. Large amount of data produced by BaBar:> 800 TB. Using the collaboration Computing Centres: –Data (re)processing (prompt reco, skimming). –Simulation production. –Data analysis  allow foreign collaborators to access data. 6 Tier A: –SLAC stores all the data + simulation: central repository. –Computing tasks divided among all the Tier A sites.

3. HEPiX Conference, SLAC, October 11th-13th 2005 3 BaBar data organization. SLACCC-IN2P3PadovaRALCNAFGridKa Data processing Data analysis  Lots of data movement + simulation production (~20 sites involved)

4. HEPiX Conference, SLAC, October 11th-13th 2005 4 Requirements. Short turn-around between data production and analysis. Data must be transfered efficiently and quickly to Computing Centres. Data transfer tools must be: –Reliable. –Robust. –Automated. –Scalable. –Easy to use.

5. HEPiX Conference, SLAC, October 11th-13th 2005 5 SLAC & CC-IN2P3. First analysis centres opened to the BaBar collaboration. Both held complete copies of data (Objectivity). Now only SLAC old a complete copy of the data. Natural candidates for testing and deployment of grid middleware. SRB: good candidate for data distribution.

6. HEPiX Conference, SLAC, October 11th-13th 2005 6 What’s SRB ? Storage Resource Broker: developed by SDSC (San Diego). Provides an uniform interface to heterogeneous storage system (disk, tape, databases) for data distributed in multiple sites. Who is using SRB ? –Biology, biomedical applications (e.g: BIRN). –Astrophysics, Earth Sciences (e.g: NASA). –Digital libraries (e.g: NARA). Used world wide: USA, Europe, Asia, Australia.

7. HEPiX Conference, SLAC, October 11th-13th 2005 7 Some SRB features. Files organized in a logical space (directories, subdirectories …): logical view totally decoupled from the physical location of the files. Replica management. Search for files based on their attributes (metadata). Fine granularity for SRB users’ACLs. Interfaced with various Mass Storage System: –Access to HPSS transparent to users. –Small files management. And many more…

8. HEPiX Conference, SLAC, October 11th-13th 2005 8 SRB architecture. 1 zone: –1 SRB/MetaCatalog server: contains list of files, physical resources, users registered. –several SRB servers to access the data at their physical location. Site 1 SRB Site 2 SRB Site 3 Application (asking for test1.txt) SRB MCAT (1) (4) (2) test1.txt (3)

9. HEPiX Conference, SLAC, October 11th-13th 2005 9 SRB BaBar architecture (2001-2004). CC-IN2P3 (Lyon) HPSS/SLAC HPSS/Lyon client SRB SLAC (Stanford, CA) SRB MCAT (1) client (3) (2) 1 Zone (SLAC)

10. HEPiX Conference, SLAC, October 11th-13th 2005 10 Extra details. Hardware: –SUN servers (Solaris 5.8, 5.9): NetraT 1405, V440. Software: –Oracle 8, 9 for the SLAC MCAT (530,000 files registered). –Multithreaded client application (Python). Advantages: –All the 3 steps controlled by the client application in //  no latency due to prestaging. –Small amount of cache needed on both sides (a few GBs). Drawbacks: –No registration into the MCAT of the files stored in Lyon. Performance: –Up to 3 TB/day from tape to tape.

11. HEPiX Conference, SLAC, October 11th-13th 2005 11 Fermilab (US)  CERN SLAC (US)  IN2P3 (FR) 1 Terabyte/day SLAC (US)  INFN Padva (IT) Fermilab (US)  U. Chicago (US) CEBAF (US)  IN2P3 (FR) INFN Padva (IT)  SLAC (US) U. Toronto (CA)  Fermilab (US) Helmholtz-Karlsruhe (DE)  SLAC (US) DOE Lab  DOE Lab SLAC (US)  JANET (UK) Fermilab (US)  JANET (UK) Argonne (US)  Level3 (US) Argonne  SURFnet (NL) IN2P3 (FR)  SLAC (US) Fermilab (US)  INFN Padva (IT) ESNET Traffic with one server on both sides (April 2004).

12. HEPiX Conference, SLAC, October 11th-13th 2005 12 SRB BaBar architecture (since 2005). CC-IN2P3 (Lyon) HPSS/Lyon SRB SLAC (Stanford, CA) SRB MCAT (1) (3) (2) HPSS/SLAC SRB MCAT 2 Zones (SLAC + Lyon)

13. HEPiX Conference, SLAC, October 11th-13th 2005 13 Extra details. Hardware: –Still the same. Software: –Oracle 10g for the SLAC MCAT. –Oracle 9i for the Lyon MCAT (migration to 10g foreseen). MCATs synchronization: only users and physical resources. Comparison of the MCATs contents to transfer the data. Advantage: –External client can pick up data from SLAC or Lyon without interacting with the other site.

14. HEPiX Conference, SLAC, October 11th-13th 2005 14 Overall assessment. A lot of time saved for developping applications thanks to the SRB. Transparent access to data: –Very useful in an hybrid environment (disk, tape). –Easy to scale the service (adding new servers). –Not dependent of physical locations changes in the application. Fully automated procedure. Easy for SLAC to recover corrupted data. 132 TB (232,000 files) shipped to Lyon. Up to 3 TB /day from tape to tape (minimum latency).

15. HEPiX Conference, SLAC, October 11th-13th 2005 15 Future plans. Possibly include RAL as part of the system: –Would reduce the network activity on the SLAC & ESNET. Some tests: –Assessment of web services technologies. SLAC HPSS/SLAC SRB HPSS/Lyon SRB CC-IN2P3 RAL SRB

16. HEPiX Conference, SLAC, October 11th-13th 2005 16 Summary and outlook. SRB used by BaBar for more than 3 years. Data shipment fully automated. Lightweight development to make it work. Easy administration. More servers on both side: –Increase peak transfer rate to 5 TB/day. –Hundreds of TB foreseen. Scalability: the sky is the limit… well and the money too… Aiming at including RAL.

17. HEPiX Conference, SLAC, October 11th-13th 2005 17 Never forget… Cannot be achieved without the work of: –The network and system admins. –The database admins. Many thanks to them!