Presentation is loading. Please wait.

Presentation is loading. Please wait.

Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David.

Published byScarlett Miller Modified over 9 years ago

Similar presentations

Presentation on theme: "Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David."— Presentation transcript:

1 Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David Smith, on behalf of IT-GT, CERN 14 Sep 2012

2 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Introduction An existing dpm-xrootd written 2006 –A pair of plugins for OFS, XMI –Only ALICE token based access –Some performance issues with dispatch of requests to dpm dpm-xrootd - 2

3 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Introduction dpm-xrootd v3 Aim to: Provide xroot file I/O for DPM for all VOs Allow participation in xrootd federations Allow ALICE token based access Other VOs to use GSI for authentication –Support VOMS extensions –Authorization is done by DPM system use of an identity (user, [/vo,/vo/group,…]). dpm-xrootd - 3

4 Grid Technology Reminder of DPM dpm-xrootd - 4 /vo /dpm /domain /home DPM head node file (uid, gid1, …) DPM disk servers DPM Name Server –Namespace –Authorization –Physical files location Disk Servers –Physical files Direct data transfer from/to disk server External transfers via gridFTP CLI, C API, SRM-enabled client, etc. data transfer

5 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Relevant features of DPM DPM has a central service –A get request for a SURL (/dpm/example.com/home/dteam/file) gives an SFN (disk001.example.com:/data1/dteam/2012-09- 14/file.1353.0) to read –A put request for a SURL returns an SFN to write to –Put concluded with put_done –Authorization check during get or put –Files can not be modified once written –DPM instance is often run for multiple VOs Disk servers providing I/O –Clients have connectivity to disk servers dpm-xrootd - 5

6 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd Now: dpm-xrootd v3 is a set of plugins to xrootd server, using plugin interfaces (as for previous version) But different interfaces: –Use XrdOss, XrdCmsClient and XrdAccAuthorize plugins VOMS extraction disabled for now dpm-xrootd - 6

7 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd - 7 Example: file access

8 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd basic file access dpm-xrootd - 8 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service

9 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd and deployment dpm-xrootd - 9 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xroot client: open xroot://dpm.example.com//dpm/example.com/dteam/file

10 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd and deployment dpm-xrootd - 10 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xroot client: open xroot://dpm.example.com//dpm/example.com/dteam/file dpm_put or dpm_get

11 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd and deployment dpm-xrootd - 11 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xroot client: open xroot://dpm.example.com//dpm/example.com/dteam/file Redirect: with host/port and opaque info &dpm.sfn=/data1/dteam/2012-09- 14/file.1353.0

12 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd and deployment dpm-xrootd - 12 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xroot client: open xroot://dpm.example.com//dpm/example.com/dteam/file Open: with original filename in the request and opaque information disk001.example.com

13 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd and deployment dpm-xrootd - 13 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xroot client: close xroot://dpm.example.com//dpm/example.com/dteam/file Open: with original filename in the request and opaque information disk001.example.com dpm_putdone (for put only)

14 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Redirection to disk server Redirection xrootd interacts with DPM –Configured to offer XrdSecunix and XrdSecgsi to the xroot client –If a valid ALICE token is present in the opaque data from the client the identity is a preset. (Check if authz or signature in opaque data) –Otherwise an identity must be derivable from the XrdSecEntity authentication data (at disk too) –Goes through get or put sequence with DPM system (currently using classic dpns/dpm api) –May need to return a wait time to the client if the get or put sequence takes more than ~1 second dpm-xrootd - 14

15 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Redirection to disk server II If get or put is successful, redirect is returned directing to the target disk server. Opaque data is added: dpm-xrootd - 15

16 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Disk access keyed hash Disk servers require keyed hash (with limited duration) in opaque data Use HMAC-SHA256-128 for keyed hash Key stored in disk and memory of xrootd processes (required to be 32 to 64 bytes) dpm-xrootd - 16

17 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT At the disk server Hash and dpm.dhost checked dpm.time must be within validity window (300 seconds default) of current time Access mode of request must be consistent with dpm.put Then disk I/O via native XrdOss ofs.persist auto hold 0 is set, to remove files not closed at the end of writing dpm-xrootd - 17

18 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT At the disk server II For Put –On success call dpm_putdone with dpm token and surl. –On fail (no close) call dpm_abortfiles with dpm token and surl. For Get –No interactions with central DPM dpm-xrootd - 18

19 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd and federation With the basic xroot file access vo/users could devise federations.. but Want to provide integrated method (as for a native xroot site) Next: a couple of example sequences dpm-xrootd - 19 FEDERATION of XROOT access

20 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd federating for VO dpm-xrootd - 20 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service

21 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd federating for VO dpm-xrootd - 21 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo)

22 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd federating for VO dpm-xrootd - 22 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo)

23 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd federating for VO dpm-xrootd - 23 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector

24 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 1 dpm-xrootd - 24 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://regional.example.org//vo/example.dat /vo/example.dat ?

25 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 1 dpm-xrootd - 25 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://regional.example.org//vo/example.dat /vo/example.dat ? XrdPss: stat /vo/example.dat (specially trusted at xrootd)

26 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 1 dpm-xrootd - 26 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://regional.example.org//vo/example.dat /vo/example.dat ? dpns_stat

27 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 1 dpm-xrootd - 27 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://regional.example.org//vo/example.dat...if site has example.dat client may be directed there Redirect to dpm.example.com:11000

28 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 1 dpm-xrootd - 28 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://regional.example.org//vo/example.dat

29 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 1 dpm-xrootd - 29 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://regional.example.org//vo/example.dat Not Involved

30 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd - 30 Client example 2

31 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 2 dpm-xrootd - 31 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://dpm.example.com//vo/example2.dat

32 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 2 dpm-xrootd - 32 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://dpm.example.com//vo/example2.dat Redirect: to dpm.example.com:11000

33 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 2 dpm-xrootd - 33 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://dpm.example.com//vo/example2.dat

34 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT federation: client example 2 dpm-xrootd - 34 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) regional redirector xroot client: open xroot://dpm.example.com//vo/example2.dat In case of no file: Redirect to regional.example.org dpm_get

35 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT dpm-xrootd - 35 Few other details

36 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT xrootd:1095 (disk server) Name2name libs dpm-xrootd - 36 xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) May use a standard name2name lib But loaded by our cmslib not OSS Calls lfn2pfn method, e.g. for global namespace to surl xrootd:11001 (fedredir_vo2) cmsd (fedredir_vo2)

37 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Plugins provided xrootd redirector –libXrdDPMFinder (XrdCmsClient) –libXrdDPMOss (XrdOss) –libXrdDPMRedirAcc (XrdAccAuthorize) xrootd disk –libXrdDPMDiskAcc (XrdAccAuthorize) –libXrdDPMOss dpm-xrootd - 37

38 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Options dpm-xrootd - 38

39 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Options: dpm dpm-xrootd - 39

40 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Options: alice token handling dpm-xrootd - 40

41 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Options: name translation dpm-xrootd - 41

42 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Status summary Available via dedicated repository –including xrootd packages (EPEL packaging) Expect xrootd 3.2.x in the EMI third party repo first –Eventually recent version of xrootd to go into EPEL dpm-xrootd setup instructions on the web –YAIM module written for yaim generation of the config files Deployments at four sites –May be others dpm-xrootd - 42

43 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT To do Eventually use dmlite rather than classic dpm/dpns api for dpm service access. VOMS dpm-xootd uses few xrootd interfaces for which ABI compatibility across minor releases isn’t guaranteed. Remove when possible. Monitoring –dpm site probably multi-vo dpm-xrootd - 43

44 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Monitoring sources dpm-xrootd - 44 xrootd:1095 (disk server) xrootd:1095 (disk server) xrootd (redirector) DPM service xrootd:11000 (fedredir_vo) cmsd (fedredir_vo) xrootd:11001 (fedredir_vo2) cmsd (fedredir_vo2)

45 CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t GT Contacts DPM Support: hep-service-dpm@cern.chhep-service-dpm@cern.ch dpm-xrootd setup wiki: –https://svnweb.cern.ch/trac/lcgdm/wiki/Dpm/Xroo t/Setuphttps://svnweb.cern.ch/trac/lcgdm/wiki/Dpm/Xroo t/Setup dpm-xrootd - 45

Download ppt "Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT dpm-xrootd v3 Creating Federated Data Stores for the LHC David."

Similar presentations

DPM Name Server (DPNS) Namespace Authorization Location of physical files DPM Server Requests queuing and processing Space Management SRM Servers v1.1,

DPM Name Server (DPNS) Namespace Authorization Location of physical files DPM Server Requests queuing and processing Space Management SRM Servers v1.1,
Environmental Council of States Network Authentication and Authorization Services The Shared Security Component February 28, 2005.

Environmental Council of States Network Authentication and Authorization Services The Shared Security Component February 28, 2005.
Report Distribution Report Distribution in PeopleTools 8.4 Doug Ostler & Eric Knapp 7264.

Report Distribution Report Distribution in PeopleTools 8.4 Doug Ostler & Eric Knapp 7264.
WebFTS as a first WLCG/HEP FIM pilot

WebFTS as a first WLCG/HEP FIM pilot
70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 7: Planning a DNS Strategy.

70-293: MCSE Guide to Planning a Microsoft Windows Server 2003 Network, Enhanced Chapter 7: Planning a DNS Strategy.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT Simplifying Configuration Ricardo Rocha ( on behalf of the LCGDM.

Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Simplifying Configuration Ricardo Rocha ( on behalf of the LCGDM.
© 2010 VMware Inc. All rights reserved Data Protection Module 10.

© 2010 VMware Inc. All rights reserved Data Protection Module 10.
LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.

LHC Experiment Dashboard Main areas covered by the Experiment Dashboard: Data processing monitoring (job monitoring) Data transfer monitoring Site/service.
TAM STE Series 2008 © 2008 IBM Corporation WebSEAL SSO, Session 108/2008 TAM STE Series 2008 - WebSEAL SSO, Session 1 Presented by: Andrew Quap.

TAM STE Series 2008 © 2008 IBM Corporation WebSEAL SSO, Session 108/2008 TAM STE Series WebSEAL SSO, Session 1 Presented by: Andrew Quap.
Summary of issues and questions raised. FTS workshop for experiment integrators Summary of use  Generally positive response on current state!  Now the.

Summary of issues and questions raised. FTS workshop for experiment integrators Summary of use  Generally positive response on current state!  Now the.
DONE-10: Adminserver Survival Tips Brian Bowman Product Manager, Data Management Group.

DONE-10: Adminserver Survival Tips Brian Bowman Product Manager, Data Management Group.
CERN - IT Department CH-1211 Genève 23 Switzerland www.cern.ch/i t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.

CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.
Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBCF GT Performant and Future Proof: MySQL, Memcache and Raspberry Pi.

Grid Technology CERN IT Department CH-1211 Geneva 23 Switzerland t DBCF GT Performant and Future Proof: MySQL, Memcache and Raspberry Pi.
Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t DBES PhEDEx Monitoring Nicolò Magini CERN IT-ES-VOS For the PhEDEx.

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES PhEDEx Monitoring Nicolò Magini CERN IT-ES-VOS For the PhEDEx.
1 Chapter Overview Publishing Resources in Active Directory Service Redirecting Folders Using Group Policies Deploying Applications Using Group Policies.

1 Chapter Overview Publishing Resources in Active Directory Service Redirecting Folders Using Group Policies Deploying Applications Using Group Policies.
CERN IT Department CH-1211 Geneva 23 Switzerland www.cern.ch/i t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, 13-14 November 2008.

CERN IT Department CH-1211 Geneva 23 Switzerland t Storageware Flavia Donno CERN WLCG Collaboration Workshop CERN, November 2008.
Configuration Management with Cobbler and Puppet Kashif Mohammad University of Oxford.

Configuration Management with Cobbler and Puppet Kashif Mohammad University of Oxford.
And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR 17.05.2012.

And Tier 3 monitoring Tier 3 Ivan Kadochnikov LIT JINR
July-2008Fabrizio Furano - The Scalla suite and the Xrootd1.

July-2008Fabrizio Furano - The Scalla suite and the Xrootd1.
DYNES Storage Infrastructure Artur Barczyk California Institute of Technology LHCOPN Meeting Geneva, October 07, 2010.

DYNES Storage Infrastructure Artur Barczyk California Institute of Technology LHCOPN Meeting Geneva, October 07, 2010.

Similar presentations

Ads by Google