RAL DCache Status Derek Ross/Steve Traylen April 2005.

Slides:



Advertisements
Similar presentations
Steve Traylen Particle Physics Department Experiences of DCache at RAL UK HEP Sysman, 11/11/04 Steve Traylen
Advertisements

DPM Basics and its status and plans Wahid Bhimji University of Edinburgh GridPP Storage Workshop – Apr 2010 Apr-101Wahid Bhimji – DPM.
Distributed Xrootd Derek Weitzel & Brian Bockelman.
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
The Zebra Striped Network Filesystem. Approach Increase throughput, reliability by striping file data across multiple servers Data from each client is.
Cs238 Lecture 3 Operating System Structures Dr. Alan R. Davis.
Oxford Jan 2005 RAL Computing 1 RAL Computing Implementing the computing model: SAM and the Grid Nick West.
The Origin of the VM/370 Time-sharing system Presented by Niranjan Soundararajan.
Data Grid Web Services Chip Watson Jie Chen, Ying Chen, Bryan Hess, Walt Akers.
CVMFS: Software Access Anywhere Dan Bradley Any data, Any time, Anywhere Project.
Event Viewer Was of getting to event viewer Go to –Start –Control Panel, –Administrative Tools –Event Viewer Go to –Start.
Test Review. What is the main advantage to using shadow copies?
Summary of issues and questions raised. FTS workshop for experiment integrators Summary of use  Generally positive response on current state!  Now the.
Chapter - 2 What is “GIT” VERSION CONTROL AND GIT BASICS.
Research on cloud computing application in the peer-to-peer based video-on-demand systems Speaker : 吳靖緯 MA0G rd International Workshop.
14 Publishing a Web Site Section 14.1 Identify the technical needs of a Web server Evaluate Web hosts Compare and contrast internal and external Web hosting.
Zhiling Chen (IPP-ETHZ) Doktorandenseminar June, 4 th, 2009.
SRM at Clemson Michael Fenn. What is a Storage Element? Provides grid-accessible storage space. Is accessible to applications running on OSG through either.
OSG Site Provide one or more of the following capabilities: – access to local computational resources using a batch queue – interactive access to local.
Data management for ATLAS, ALICE and VOCE in the Czech Republic L.Fiala, J. Chudoba, J. Kosina, J. Krasova, M. Lokajicek, J. Svec, J. Kmunicek, D. Kouril,
© 2007 Cisco Systems, Inc. All rights reserved.Cisco ConfidentialPresentation_ID 1 Partitions & Search Spaces TOI Aaron Belcher.
Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 3: Operating-System Structures System Components Operating System Services.
D C a c h e Michael Ernst Patrick Fuhrmann Tigran Mkrtchyan d C a c h e M. Ernst, P. Fuhrmann, T. Mkrtchyan Chep 2003 Chep2003 UCSD, California.
Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.
Bulk Data Movement: Components and Architectural Diagram Alex Sim Arie Shoshani LBNL April 2009.
Andrew C. Smith – Storage Resource Managers – 10/05/05 Functionality and Integration Storage Resource Managers.
Steve Traylen PPD Rutherford Lab Grid Operations PPD Christmas Lectures Steve Traylen RAL Tier1 Grid Deployment
Redirector xrootd proxy mgr Redirector xrootd proxy mgr Xrd proxy data server N2N Xrd proxy data server N2N Global Redirector Client Backend Xrootd storage.
Chapter 10 Chapter 10: Managing the Distributed File System, Disk Quotas, and Software Installation.
CE Operating Systems Lecture 13 Linux/Unix interprocess communication.
1 LHCb on the Grid Raja Nandakumar (with contributions from Greig Cowan) ‏ GridPP21 3 rd September 2008.
USATLAS dCache System and Service Challenge at BNL Zhenping (Jane) Liu RHIC/ATLAS Computing Facility, Physics Department Brookhaven National Lab 10/13/2005.
Jens G Jensen RAL, EDG WP5 Storage Element Overview DataGrid Project Conference Heidelberg, 26 Sep-01 Oct 2003.
Derek Ross E-Science Department DCache Deployment at Tier1A UK HEP Sysman April 2005.
Making SIP NAT Friendly Jonathan Rosenberg dynamicsoft.
2-Sep-02Steve Traylen, RAL WP6 Test Bed Report1 RAL and UK WP6 Test Bed Report Steve Traylen, WP6
BNL Service Challenge 3 Status Report Xin Zhao, Zhenping Liu, Wensheng Deng, Razvan Popescu, Dantong Yu and Bruce Gibbard USATLAS Computing Facility Brookhaven.
The CMS Top 5 Issues/Concerns wrt. WLCG services WLCG-MB April 3, 2007 Matthias Kasemann CERN/DESY.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Upcoming Features and Roadmap Ricardo Rocha ( on behalf of the.
Data Transfer Service Challenge Infrastructure Ian Bird GDB 12 th January 2005.
FTS monitoring work WLCG service reliability workshop November 2007 Alexander Uzhinskiy Andrey Nechaevskiy.
© Copyright 2004 Instrumental, Inc I/O Types and Usage in DoD Henry Newman Instrumental, Inc/DOD HPCMP/DARPA HPCS May 24, 2004.
SESEC Storage Element (In)Security hepsysman, RAL 0-1 July 2009 Jens Jensen.
LCG Storage Workshop “Service Challenge 2 Review” James Casey, IT-GD, CERN CERN, 5th April 2005.
CernVM-FS Infrastructure for EGI VOs Catalin Condurache - STFC RAL Tier1 EGI Webinar, 5 September 2013.
ASCC Site Report Eric Yen & Simon C. Lin Academia Sinica 20 July 2005.
BNL dCache Status and Plan CHEP07: September 2-7, 2007 Zhenping (Jane) Liu for the BNL RACF Storage Group.
LHCC Referees Meeting – 28 June LCG-2 Data Management Planning Ian Bird LHCC Referees Meeting 28 th June 2004.
1 5/4/05 Fermilab Mass Storage Enstore, dCache and SRM Michael Zalokar Fermilab.
J Jensen/J Gordon RAL Storage Storage at RAL Service Challenge Meeting 27 Jan 2005.
The RAL PPD Tier 2/3 Current Status and Future Plans or “Are we ready for next year?” Chris Brew PPD Christmas Lectures th December 2007.
INFSO-RI Enabling Grids for E-sciencE File Transfer Software and Service SC3 Gavin McCance – JRA1 Data Management Cluster Service.
Andrea Manzi CERN EGI Conference on Challenges and Solutions for Big Data Processing on cloud 24/09/2014 Storage Management Overview 1 24/09/2014.
Embedded Real-Time Systems Processing interrupts Lecturer Department University.
Martina Franca (TA), 07 November Installazione, configurazione, testing e troubleshooting di Storage Element.
An Analysis of Data Access Methods within WLCG Shaun de Witt, Andrew Lahiff (STFC)
GGUS summary (3 weeks) VOUserTeamAlarmTotal ALICE7029 ATLAS CMS LHCb Totals
TNPM v1.3 Flow Control. 2 High Level Instead of each component having flow control settings that govern only its directory, we now have a set of flow.
DB Questions and Answers open session (comments during session) WLCG Collaboration Workshop, CERN Geneva, 24 of April 2008.
Patrick Gartung 1 CMS 101 Mar 2007 Introduction to the User Analysis Facility (UAF) Patrick Gartung - Fermilab.
Open Science Grid Consortium Storage on Open Science Grid Placing, Using and Retrieving Data on OSG Resources Abhishek Singh Rana OSG Users Meeting July.
Bullet Proof FTP Server © N. Ganesan, Ph.D.. Contributions Steven Fong Jennifer Abeywardena Phoung Co ChengYu Wang.
dCache Paul Millar, on behalf of the dCache Team
CASTOR: possible evolution into the LHC era
Service Challenge 3 CERN
Bernd Panzer-Steindel, CERN/IT
Artem Trunov and EKP team EPK – Uni Karlsruhe
lundi 25 février 2019 FTS configuration
Software - Operating Systems
The LHCb Computing Data Challenge DC06
Presentation transcript:

RAL DCache Status Derek Ross/Steve Traylen April 2005

Outline Proposed setup for production system. Multi-homing for SC2. Issues, problems and confusions. DCache Errors?

Questions about Production Setup Is this sensible, any better ideas. CMS and others look to be assuming they can delete files immediately. –This includes files on tapes. –There is no delete method to interact with the HSM. Truncate option does not work for us? We don’t know how to drain a disk server for maintenance.

Quotas on Production Setup Currently we quota VOs by having pools uniquely attracted to say. –/pnfs/gridpp.rl.ac.uk/data/dteam This should allow us to drain a disk partition and move it to a different VO to change allocations. We can also have a shared pool to introduce soft limits. (The tape backed disk will be like this) But we reduce the number of servers that can do I/O though. Anything better? Storage space advertised needs to reflect this.

Multi Homing for SC2 Aim –Have disk servers on production network. These were running the pool software. –Expose this to UK light network via multi-homed GridFTP servers. –We could not get it to work. FTP protocol returned i.p. for data channel on wrong interface. (Can we bind the gridFTP doors?) Pool nodes tried to contact external interfaces of GridFTP Nodes. For SC3 this problem may go away but is very likely relevent for other T2 sites.

Data Flow We have general confusion about what happens with. –SRMGet –SRMPut –SRMCopy (Push and Pull) In particular what initiates the various connections at GridFTP and dCap levels. Choice of GridFTP server is not influenced by location of data?

DCache Errors Some errors encountered during running of dCache –All seen in latest release –Full java backtraces omitted for brevity –None appear to affect operation, but disconcerting at first –Does make it harder to see actual errors in logs –Ignoring errors is not a good habit to get into

Odd Errors 1 Extra blank line in.poollist leads to following in Domain.log : CellAdapter : prepareRemoval : got java.lang.NullPointerException …

Odd Errors 2 From pnfs.log : Error obtaining 'l' flag for getSimulatedFilesize : java.io.FileNotFoundException: /pnfs/fs/.(puse)( )(2) (Is a directory)

Odd Errors 3 From srm.log during startup : java.sql.SQLException: ERROR: Relation 'srmnextrequestid' already exists : at org.postgresql.core.QueryExecutor.executeV2(QueryExec utor.java:287) …

Odd Errors 4 From gridftpdoor.log for every transfer : java.io.IOException: Log root directory /tmp/dcache- ftp-tlog not found : at diskCacheV111.doors.FTPTransactionLog.begin(FTPTransa ctionLog.java:98) : at diskCacheV111.doors.GsiFtpDoorV1.startTlog(GsiFtpDoor V1.java:109) … –Does go away if directory is created

Odd Errors 5 From http.log: : Exception in Loop : java.lang.ClassCastException From dcache.log: : java.io.InvalidClassException: dmg.cells.nucleus.CellMessage; local class incompatible: stream classdesc serialVersionUID = , local class serialVersionUID = : at java.io.ObjectStreamClass.initNonProxy(Unknown Source) …