Storage and Data Movement at FNAL D. Petravick CHEP 2003

3/25/2003DLP -- CHEP FNAL Overall Goals Provide competent one copy permanent store to all the Lab’s experiments. Provide scalable and performant data flows –Tape and Disk –Local area and Wide area. Provide standard interfaces allowing for interoperation with other sites. Collaborate! (DESY > 5 yrs) GLOBUS, LBL, JLAB, CERN

3/25/2003DLP -- CHEP 20033

3/25/2003DLP -- CHEP Overview Central Storage Systems capacious and scalable enough to be a hub of a data intensive system. –Linux as a hardware platform. –Permanent and temporary semantics. Competent local and GRID interfaces. ENCP, dCap FTP(s) and SRM (Storage Resource Manager) investigating GLUE schema and monitoriing

3/25/2003DLP -- CHEP 20035

3/25/2003DLP -- CHEP 20036

3/25/2003DLP -- CHEP 20037

3/25/2003DLP -- CHEP 20038

3/25/2003DLP -- CHEP RunII Work Plan SAM –Grid FTP integration with SAM. –DCAP integration –SRM integration with SAM. SRM integration with legacy CDF AC++ Restore rates for CDF experiment. Provide hardware for D0 Wide area transfers.

3/25/2003DLP -- CHEP US CMS Work Plan Use dCache as a vehicle for Grid File access locally. Evaluate Storage elements for Tier II centers (dCache, DRM, NeST EDG wp 5, Dfarm). Follow, help formulate LCG requirements. Detailed work to meet CMS Data Challenge on CMS deadlines. Monitor and Improve Network –Pinger IEPM (co-project) –FAST and other h/p TCP stacks. (co-project)

3/25/2003DLP -- CHEP

3/25/2003DLP -- CHEP Job-Aware, Replica-Aware Mdw Permanent Store Replica Stores FTP Inter- face SRM Inter- face Direct File Access Interface Monitoring Interface Framework + job

3/25/2003DLP -- CHEP Lattice Gauge Work Plan Support the Fermilab Lattice Computation Facility. Use GRID FTP, SRM to integrate with –JLAB (cluster), BNL (QCDOC) facilities. Understand relationship of storage systems to QIO (community I/O package) Investigate utility of caching data at FNAL. Investigate symmetries with earth science type system.

3/25/2003DLP -- CHEP Other Work Elements Work on Grid FTP V1.1 specification. –Include work on a few scaling issues for “seas of linux box” type systems. Work on SRM protocol, futures. –V2.1 (co – FNAL) Look at “Object Based Storage” in conjunction with US CMS. –Proposed investigation is root integration. Understand the requirements of analysis systems.

3/25/2003DLP -- CHEP Other work elements… Support Grid protocols for dFarm. –Make it a “storage element” –Another innovative package. Experimenting with Grid Authentication. Prepare for special routing on the WAN –E.g. lambda networking. Support the experimental community at FNAL. –Auger in GridFTP production. –Minos in Kerberized, Weak FTP.

3/25/2003DLP -- CHEP

3/25/2003DLP -- CHEP Advanced Monitoring Animations were Crucial for debugging The RUN II LAN based data systems. Looking for analogous tools for grid based works Follow items such as GLUE.

3/25/2003DLP -- CHEP Advanced Network Integration FNAL proposes to have two paths off site. Path to Starlight will allow R&D on advanced network concepts (e.g. lambda network) Central data movement systems will be an early user.

3/25/2003DLP -- CHEP Summary Very substantial and successful support of FNAL Program. Very proactive work on fabric side. Data in Movement are interesting. Goal :Routine –performant flows –on LAN and WAN –with interoperation