CERN IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t DPM status and plans David Smith CERN, IT-DM-SGT Pre-GDB, Grid Storage Services 11 November.

Slides:



Advertisements
Similar presentations
Data & Storage Services CERN IT Department CH-1211 Genève 23 Switzerland t DSS CASTOR Status Alberto Pace.
Advertisements

DPM Basics and its status and plans Wahid Bhimji University of Edinburgh GridPP Storage Workshop – Apr 2010 Apr-101Wahid Bhimji – DPM.
DPM Name Server (DPNS) Namespace Authorization Location of physical files DPM Server Requests queuing and processing Space Management SRM Servers v1.1,
Storage: Futures Flavia Donno CERN/IT WLCG Grid Deployment Board, CERN 8 October 2008.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David.
CERN - IT Department CH-1211 Genève 23 Switzerland t SVN Pilot: CVS Replacement Manuel Guijarro Jonatan Hugo Hugosson Artur Wiecek David.
Operating Systems & Infrastructure Services CERN IT Department CH-1211 Geneva 23 Switzerland t OIS CERN Search Updates Eduardo Alvarez November.
CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
LHCC Comprehensive Review – September WLCG Commissioning Schedule Still an ambitious programme ahead Still an ambitious programme ahead Timely testing.
DPM CCRC - 1 Research and developments DPM status and plans Jean-Philippe Baud.
CERN IT Department CH-1211 Genève 23 Switzerland t Service Management GLM 15 November 2010 Mats Moller IT-DI-SM.
CERN IT Department CH-1211 Genève 23 Switzerland t Plans and Architectural Options for Physics Data Analysis at CERN D. Duellmann, A. Pace.
CERN IT Department CH-1211 Genève 23 Switzerland t EIS section review of recent activities Harry Renshall Andrea Sciabà IT-GS group meeting.
SRM 2.2: status of the implementations and GSSD 6 th March 2007 Flavia Donno, Maarten Litmaath INFN and IT/GD, CERN.
CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Monitoring for the LHC experiments Irina Sidorova (CERN, JINR) on.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES PhEDEx Monitoring Nicolò Magini CERN IT-ES-VOS For the PhEDEx.
CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
CERN IT Department CH-1211 Genève 23 Switzerland t Tier0 Status - 1 Tier0 Status Tony Cass LCG-LHCC Referees Meeting 18 th November 2008.
The huge amount of resources available in the Grids, and the necessity to have the most up-to-date experimental software deployed in all the sites within.
Introduction to dCache Zhenping (Jane) Liu ATLAS Computing Facility, Physics Department Brookhaven National Lab 09/12 – 09/13, 2005 USATLAS Tier-1 & Tier-2.
CERN IT Department CH-1211 Genève 23 Switzerland t Castor development status Alberto Pace LCG-LHCC Referees Meeting, May 5 th, 2008 DRAFT.
CERN IT Department CH-1211 Genève 23 Switzerland t Monitoring: Tracking your tasks with Task Monitoring PAT eLearning – Module 11 Edward.
Enabling Grids for E-sciencE Introduction Data Management Jan Just Keijser Nikhef Grid Tutorial, November 2008.
Light weight Disk Pool Manager experience and future plans Jean-Philippe Baud, IT-GD, CERN September 2005.
Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Usage of virtualization in gLite certification Andreas Unterkircher.
Owen SyngeTitle of TalkSlide 1 Storage Management Owen Synge – Developer, Packager, and first line support to System Administrators. Talks Scope –GridPP.
INFSO-RI Enabling Grids for E-sciencE Enabling Grids for E-sciencE Pre-GDB Storage Classes summary of discussions Flavia Donno Pre-GDB.
CERN - IT Department CH-1211 Genève 23 Switzerland t OIS Deployment of Exchange 2010 mail platform Pawel Grzywaczewski, CERN IT/OIS HEPIX.
WLCG Grid Deployment Board, CERN 11 June 2008 Storage Update Flavia Donno CERN/IT.
CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Job Priorities update Andrea Sciabà IT/GS Ulrich Schwickerath IT/FIO.
CERN IT Department CH-1211 Genève 23 Switzerland t Load Testing Dennis Waldron, CERN IT/DM/DA CASTOR Face-to-Face Meeting, Feb 19 th 2009.
Derek Ross E-Science Department DCache Deployment at Tier1A UK HEP Sysman April 2005.
EGEE-III INFSO-RI Enabling Grids for E-sciencE Ricardo Rocha CERN (IT/GS) EGEE’08, September 2008, Istanbul, TURKEY Experiment.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT DPM / LFC and FTS news Ricardo Rocha ( on behalf of the IT/GT/DMS.
DPM Python tools Ivan Calvet IT/SDC-ID DPM Workshop 10 th October 2014.
INFSO-RI Enabling Grids for E-sciencE Introduction Data Management Ron Trompert SARA Grid Tutorial, September 2007.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES GGUS Ticket review T1 Service Coordination Meeting 2010/10/28.
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
The CMS Top 5 Issues/Concerns wrt. WLCG services WLCG-MB April 3, 2007 Matthias Kasemann CERN/DESY.
Data & Storage Services CERN IT Department CH-1211 Genève 23 Switzerland t DSS XROOTD news New release New features.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Upcoming Features and Roadmap Ricardo Rocha ( on behalf of the.
CERN IT Department CH-1211 Genève 23 Switzerland t HEPiX Conference, ASGC, Taiwan, Oct 20-24, 2008 The CASTOR SRM2 Interface Status and plans.
Operating Systems & Information Services CERN IT Department CH-1211 Geneva 23 Switzerland t OIS Drupal at CERN Juraj Sucik Jarosław Polok.
23 January 2007WLCG workshop, CERN System Management Working Group Alessandra Forti WLCG workshop CERN, 23 January 2007.
INFSO-RI Enabling Grids for E-sciencE SRMv2.2 in DPM Sophie Lemaitre Jean-Philippe.
1Maria Dimou- cern-it-gd LCG November 2007 GDB October 2007 VOM(R)S Workshop report Grid Deployment Board.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Overview of DMLite Ricardo Rocha ( on behalf of the LCGDM team.
WLCG Grid Deployment Board CERN, 14 May 2008 Storage Update Flavia Donno CERN/IT.
CERN - IT Department CH-1211 Genève 23 Switzerland t Operating systems and Information Services OIS Proposed Drupal Service Definition IT-OIS.
Enabling Grids for E-sciencE INFSO-RI Enabling Grids for E-sciencE Gavin McCance GDB – 6 June 2007 FTS 2.0 deployment and testing.
CERN - IT Department CH-1211 Genève 23 Switzerland CASTOR F2F Monitoring at CERN Miguel Coelho dos Santos.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Data Management cluster summary David Smith JRA1 All Hands meeting, Catania, 7 March.
CMS: T1 Disk/Tape separation Nicolò Magini, CERN IT/SDC Oliver Gutsche, FNAL November 11 th 2013.
Grid Deployment Board 5 December 2007 GSSD Status Report Flavia Donno CERN/IT-GD.
Site Authorization Service Local Resource Authorization Service (VOX Project) Vijay Sekhri Tanya Levshina Fermilab.
The Grid Storage System Deployment Working Group 6 th February 2007 Flavia Donno IT/GD, CERN.
SRM 2.2: experiment requirements, status and deployment plans 6 th March 2007 Flavia Donno, INFN and IT/GD, CERN.
CERN - IT Department CH-1211 Genève 23 Switzerland t Service Level & Responsibilities Dirk Düllmann LCG 3D Database Workshop September,
DPM: Future Proof Storage Ricardo Rocha ( on behalf of the DPM team ) EMI INFSO-RI
CERN - IT Department CH-1211 Genève 23 Switzerland t ASM and Oracle Service Availability Monitoring LCG 3D Workshop CERN, January 26 th,
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Standard Protocols in DPM Ricardo Rocha.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Update on Service Availability Monitoring (SAM) Marian Babik, David Collados,
CERN IT Department CH-1211 Genève 23 Switzerland t CMS SAM Testing Andrea Sciabà Grid Deployment Board May 14, 2008.
EMI is partially funded by the European Commission under Grant Agreement RI Future Proof Storage with DPM Oliver Keeble (on behalf of the CERN IT-GT-DMS.
LCG Introduction John Gordon, STFC GDB December 7 th 2010.
EGEE Data Management Services
Status of the SRM 2.2 MoU extension
StoRM Architecture and Daemons
SRM2 Migration Strategy
Data Management cluster summary
Presentation transcript:

CERN IT Department CH-1211 Genève 23 Switzerland t DPM status and plans David Smith CERN, IT-DM-SGT Pre-GDB, Grid Storage Services 11 November 2008

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Overview DPM –Current status –Forthcoming release –Plans Summary DPM status and plans - 2

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services DPM current status DPM is the last stable release on SLC4 (since 18 Aug) Issues are reported and handled via GGUS, Savannah, support and user-forum mailing lists DPM is used by about 187 sites in LCG –Current LCG statistics: DPM status and plans - 3

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services DPM v1.7.0 The next release is expected to be v1.7.0 and is currently in certification. Changes include: srmCopy has been fully implemented Additions for the WLCG SRM MoU addendum: –write permission to a space can be limited to multiple groups –SRM2.2 srmReleaseFiles can be used without a requestToken –srmLs can return the list of spaces a given file is in A new dpm-listspaces command line tool which also serves as a information provider has been added (a contribution from Michel Jouvin) DPM retains space token when the admin drains a file from a filesystem (as a part of maintenance) DPM status and plans - 4

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services DPM v1.7.0 Added ADLER32 and CRC32 to the supported checksum types for the gridftp CKSM command –Server side checksum calculation of a file already in the DPM New nameserver API calls to set or retrieve checksums Added SURL cleanup after expiry or abort in some circumstances Made some changes in preparation for SLC5 Some more internal logging for space capacity updates Version will require an update of the dpm_db database schema –YAIM will do the update automatically. As usual a database backup beforehand is recommended. DPM status and plans - 5

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services DPM information provider Version will use dpm-listspaces to provide more information about spaces A gLite patch was also created to add dpm-listspaces to the current DPM version –Entered certification before the DPM release, may be available as a gLite update before the new DPM release Updates glite-yaim-dpm, glite-info-dynamic-dpm, DPM- interfaces Adds glite-info-provider-service –Will require YAIM to be rerun to setup the new providers but no changes in site-info.def needed DPM status and plans - 6

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services DPM plans A more friendly DPM python interface has been developed –Not yet released, decided to add to gLite as an entirely new package separate from the main DPM bundle For DPM the following are areas of interest Enhance xrootd support –Pass xrootd client authentication details to the DPM for authorization decisions Quota system Further checksum support –Calculation on demand is already in gridftp2 DPM status and plans - 7

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services DPM plans Monitoring interface –To provide feedback on the state of the system Tool for consistency check and repair –Both internal DPM structures and possible namespace verification by users Enhance filesystem selection Investigate a shared filesystem interface (e.g. NFS 4.1) –Input from the T1/T2 analysis working group will be used to decide direction Resource limits –Service resource usage and disk server I/O limits Compliance with the gLite logging recommendations –In particular logging via syslog for authentication events Must also remain alert to the issues arising from sites and experiment requests DPM status and plans - 8

CERN IT Department CH-1211 Genève 23 Switzerland t Internet Services Summary New release of DPM is currently in certification –There may be an update of the information provider for the current DPM version beforehand Have shown areas of interest for DPM development –But must balance with the requests and issues arising in the significant deployment base DPM status and plans - 9