1. 1 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Lawrence Berkeley National Laboratory June 2, 2000 CDIC Seminar

2. 2 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Outline The Grid Grand Challenge NOVA PPDG Summary

3. 3 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Vision Today: –you don’t care which power plant actually produced the electric power –you just plug your microwave and use it Tomorrow: –you don’t care which actual computer did your computations –you just plug your PDA and use it

4. 4 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Virtual Data Grid OO Computing –encapsulation of data and algorithms –large data transfers Selection Parameters Federation DB1 DB3 DB4 DB5 DB6 CPU Local Remote OODB DB2 Drawing by Federico Carminati

5. 5 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Algorithmic Virtual Data Agent Computing –separation of data and algorithms –small data transfers Algorithmic Virtual Data Selection Parameters DB1 DB4 DB5 DB6 CPU Local Remote Procedure Proc.C PROOF CPU TagDB RDB DB3 DB2 Drawing by Federico Carminati

6. 6 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Data Access –GCA: Grand Challenge Architecture http://www-rnc.lbl.gov/GC optimized storage access for physics analysis integrating with the STAR experiment at RHIC –NOVA: Networked Object-based enVironment for Analysis http://duvall.star.bnl.gov/nova prototyped components for physics analysis in distributed environment –PPDG: Particle Physics Data Grid http://www.cacr.caltech.edu/ppdg addresses long-term data-management needs of high-energy and nuclear physics rapidly deploys advanced services

7. 7 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine GCA Participants NERSC/Berkeley Lab –L. Bernardo, A. Mueller, H. Nordberg, A.Shoshani, A. Sim, J. Wu Argonne –D. Malon, E. May, G. Pandola Brookhaven Lab –B. Gibbard, S. Johnson, J. Porter, T.Wenaus Nuclear Science/Berkeley Lab –D. Olson, A. Vaniachine, J. Yang, D.Zimmerman

8. 8 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Grand Challenge Architecture An order-optimized prefetch architecture for data retrieval from multilevel storage in a distributed multiuser environment Queries select events and specific event components based upon tag attribute ranges –query estimates are provided prior to execution –support for collections as queries –Because event components are distributed over several files, processing an event requires delivery of a “bundle” of files Events are delivered in an order that takes advantage of what is already on disk, and multi-user policy-based prefetching of further data from tertiary storage GCA inter-component communication is though a CORBA layer (shielded from physicists)

9. 9 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Challenge There are several –Not all data fits on disk ($$) Part of 1 year’s DST’s fit on disk –What about last year, 2 year’s ago? –What about hits, raw? –Available disk bandwidth means data read into memory must be efficiently used ($$) don’t read unused portions of the event Don’t read events you don’t need –Available tape bandwidth means files read from tape must be shared by many users, files should not contain unused bytes ($$$$) –Facility resources are sufficient only if used efficiently Should operate steady-state (nearly) fully loaded

10. 10 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Bottleneks Keep recently accessed data on disk, but manage it so unused data does not waste space. Try to arrange that 90% of file access is to disk and only 10% are retrieved from tape.

11. 11 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Efficient Access Split event into components across different files so that most bytes read are used –Raw, tracks, hits, tags, summary, trigger, … Optimize file size so tape bandwidth is not wasted –1GB files,  means different # of events in each file Coordinate file usage so tape access is shared –Users select all files at once –System optimizes retrieval and order of processing Use disk space & bandwidth efficiently –Operate disk as cache in front of tape

12. 12 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Bundles Event Identifiers (Run#, Event#) Event components Files File bundle 1File bundle 2File bundle 3

13. 13 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Event Components T. Ullrich, Jan. 2000

14. 14 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine STACS: STorage Access Coordination System Bit-Sliced Index File Catalog Policy Module Query Status, CacheMap Query Monitor List of file bundles and events Cache Manager Requests for file caching and purging Query Estimator Estimate pftp and file purge commands File Bundles, Event lists Query

15. 15 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine database Data Flow GC System StIOMaker fileCatalog tagDB Query Monitor Cache Manager Query Estimator STAR Software Index Builder gcaClient FileCatalog IndexFeeder GCA Interface

16. 16 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Eliminating Dependencies StIOMaker ROOT + STAR Software StChallenger ::Challenge() libGCAClient.so libChallenger.so (implementation) CORBA + GCA software libOB.so TNamed > StFileI

17. 17 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine GCA-STAR Integration Workshop http://www-rnc.lbl.gov/GC/meetings/14mar00/default.htm 14-15 March 2000

18. 18 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine GCA Services Status Running on rsun00 since end of March 3.6 TB of simulated data indexed 179 physics tags: –StrangeTag –FlowTag –ScaTag 240 user queries served Next week: index of 1M cosmic events

19. 19 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine NOVA Networked Object-based EnVironment for Analysis Modular components for distributed computing; application-neutral interfaces –Can be used as a coherent framework or in isolation to extend existing analysis systems Focused on support for C++ based analysis –Used for all RHIC, LHC, other large experiments Emphasis on user participation in iterative development; real-world prototyping and testing Use of existing tools and technologies

20. 20 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Architecture

21. 21 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Domains –Regional Center Central management and execution of analysis –Remote Client Mobile Analysis –Middleware Components Data exchange and navigation tools Client/Server object request brokerage –Data Management Data repository, catalogue, and interface Data model for simple objects (C-structs)

22. 22 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Deliverables Configuration manager for analysis jobs Analysis workflow catalog Database for versioned dataObjects Web-based database navigation tool Distributed job submission and monitoring system

23. 23 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Job monitoring system Workflow Cataloguing fileCatalog Job configuration manager

24. 24 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine PPDG Participants

25. 25 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Data Replication LBNL - ESNet: 612Mb/s (OC12) BNL - ESNet: 45Mb/s (T3, does 1 MB/s LBNL/NERSC RCF) BNL - ESNet: 145Mb/s after upgrade to OC3 (local: 100 Mb/s, firewall: 70Mb/s) STAR: BNL to LBNL peak: 3 MB/s sustained: 0.1 TB/day Planned for July 2000 PPDG: ANL to LBNL OC3: 3 MB/s OC12: 8 MB/s

26. 26 Efficient Data Access for Distributed Computing at RHIC A. Vaniachine Summary Grand Challenge –optimized access to multi-component event data files stored in HPSS –tested up to 10M events, 250 concurrent queries –has been deployed in the STAR experiment –solutions for data management in PPDG project NOVA –developed tools for distributed physics analysis –technologies for “Algrorithmic Virtual Data” LDRD proposal