Presentation is loading. Please wait.

Presentation is loading. Please wait.

БАЗОВЫЕ СРЕДСТВА РАБОТЫ С ФАЙЛАМИ В GRID. The gLite3 Architecture Security Service: Grid Security Infrastructure (GSI) Secure Sockets Layer (SSL) communication.

Similar presentations


Presentation on theme: "БАЗОВЫЕ СРЕДСТВА РАБОТЫ С ФАЙЛАМИ В GRID. The gLite3 Architecture Security Service: Grid Security Infrastructure (GSI) Secure Sockets Layer (SSL) communication."— Presentation transcript:

1 БАЗОВЫЕ СРЕДСТВА РАБОТЫ С ФАЙЛАМИ В GRID

2 The gLite3 Architecture Security Service: Grid Security Infrastructure (GSI) Secure Sockets Layer (SSL) communication protocol IS - Information Service: 1.Globus Monitoring and Discovery Service (MDS) discovery and to publish the resource status 2. Relational Grid Monitoring Architecture (R-GMA) accounting, monitoring and publication Data Management: SE - Storage Element, Protocols Catalogues File Transfer SRM Workload Management Services : Workload Management System (WMS) Local Resource Management System (batch) Worker Nodes (WNs) UI - User Interface : list all the resources submit/cancel/retrieve jobs, output, status, logging … copy, replicate and delete files from the Grid retrieve the status of different resources from the IS

3 Начало работы UI = lxpub04.jinr.ru lxpub01.jinr.ru, lxpub02.jinr.ru, lxpub03.jinr.ru ssh lxpubXX.jinr.ru source /opt/ui/external/etc/profile.d/grid- env.sh *Standard Proxy ($X509_USER_PROXY =/tmp/x509up_ ) lcgce01://etc/grid_security/grid_mapfile Configuration file: $EDG_LOCATION=/opt/ui/edg $GLITE_LOCATION=/opt/ui/glite lxpub03:~ > grid-proxy-init lxpub03:~ > grid-proxy-info lxpub03:~ > grid-proxy-destroy

4 *VOMS Proxies (Standard Grid proxy$X509_USER_PROXY=/tmp/x509up_u + full VOMS proxy) Configuration file: $GLITE_LOCATION=/opt/ui/glite $GLITE_LOCATION/etc/vomses, /opt/glite/etc/vomses $ voms-proxy-init --voms : [Role= ] lxpub03:~ > voms-proxy-init --voms atlas lxpub03:~ > voms-proxy-info -all lxpub03:~ > voms-proxy-destroy

5 *Creating a long-term proxy and storing it in MyProxy Server $ myproxy-init -s -d -n, where : -s =MyProxy Server -d option instructs the server to associate the user DN to the proxy lxpub03:~ > echo $MYPROXY_SERVER lcgpx01.jinr.ru lxpub03:~ > myproxy-init -s lcgpx01 -d –n Your identity: /C=RU/O=RDIG/OU=users/OU=jinr.ru/CN=Natalia Gromova Enter GRID pass phrase for this identity: Creating proxy...................................... Done Proxy Verify OK Your proxy is valid until: Thu Apr 17 17:09:55 2008 A proxy valid for 168 hours (7.0 days) for user /C=RU/O=RDIG/OU=users/OU=jinr.ru/ lxpub03:~ > myproxy-destroy -s lcgpx01 -d Default MyProxy credential for user /C=RU/O=RDIG/OU=users/OU=jinr.ru/CN=Natalie gromova was successfully removed.

6 Information Service (IS) Two IS systems are used in gLite 3.1 : 1. MDS - Globus Monitoring and Discovery Service : 2. Relational Grid Monitoring Architecture (R- GMA)

7 MDS - Globus Monitoring and Discovery Service : -lcg-infosites (VO-specific information on existing Grid resources ) - lcg-info (list either CEs or SE) - Local GRIS (The first level of MDS information publication is the GRIS, provides specific information for a particular service) -Site BDII (collects information about all resources present at a site i.e. data from all GRISes at the site ) Relational Grid Monitoring Architecture (R-GMA) https://lcgmon01.gridpp.rl.ac.uk:8443/R-GMA/index.html

8 lcg-infosites Command can be used to obtain VO-specific information on existing Grid resources. Syntax: lcg-infosites --vo -v -f --is -vo : the name of the VO : -ce: the number of CPUs, running jobs, waiting jobs and CE names -v 1: only the CE names; -v 2: the cluster names, the amount of RAM, the operating system name and version and the processormodel; -se: the names of the SEs supporting the VO, the type of storage system and the used and available space; -v 1: only the SE names; all: the information given by ce and se; together; -closeSE: the names of the CEs supporting the VO and their close SEs; -tag: the software tags published by each CE supporting the VO;

9 -lfc: the hostname of the LFC catalogues available to the VO; -lfcLocal: the hostname of the local LFC catalogues available to the VO; -rb: the hostname and port of the RBs available to the VO; -dli: the Data Location Index servers available to the VO; -dliLocal: the local Data Location Index servers available to the VO; -vobox: the VO boxes available to the VO; -fts: the endpoints of the FTS servers available to the VO; -sitenames:the names of all WLCG/EGEE sites; -is :the BDII to query ($LCG_GFAL_INFOSYS=lcgbdii.jinr.ru:2170) -f : the information printed to the specified site

10 Example - Obtaining information about computing resources lxpub03:~ > lcg-infosites --vo atlas ce valor del bdii: lcgbdii.jinr.ru:2170 #CPU Free Total Jobs Running Waiting ComputingElement ---------------------------------------------------------- 40 18 0 0 0 ce.ui.savba.sk:2119/jobmanager-pbs-atlas 330 83 4 2 2 lcgce01.jinr.ru:2119/jobmanager-lcgpbs-atlas 330 83 4 2 2 lcgce02.jinr.ru:2119/jobmanager-lcgpbs-atlas lxpub03:~ > lcg-infosites --vo atlas ce | grep jinr valor del bdii: lcgbdii.jinr.ru:2170 11 8 2 0 2 lgdce01.jinr.ru:2119/jobmanager-lcgpbs-atlas

11 Example - Obtaining information about storage resources lxpub03:~ > lcg-infosites --vo atlas se Avail Space(Kb) Used Space(Kb) Type SEs ------------------------------------------------------------------------------------------ 14462833655808 2537166995456 n.a storm-fe.cr.cnaf.infn.it n.a 999999 n.a atlasse01.ihep.ac.cn 30863580252 12578843986 n.a lxfs07.jinr.ru …. Example - Host name of LFC the catalogues VO lxpub03:~ > lcg-infosites --vo atlas lfc prod-lfc-atlas-central.cern.ch Example - Listing local LFC servers lxpub03:~ > lcg-infosites --vo atlas lfcLocal lfc.ihep.ac.cn lcg04.usatlas.bnl.gov prod-lfc-atlas-local.cern.ch.... lxpub03:~ > lcg-infosites --vo atlas lfcLocal | grep jinr lcglfc01.jinr.ru

12 lcg-info The lcg-info command can be used to list CEs / Ses satisfying a given set of conditions on their attributes. The information is taken from the BDII specified by the $LCG_GFAL_ INFOSYS environment variable. Syntax: lcg-info [--list-ce | --list-se] [--query ] [-- attrs ] where: --list-ce,--list-se must be used to indicate if CEs or SEs should be listed. --query option introduces a filter (conditions to be fulfilled) to the elements of the list --attrs option may be used to specify which attributes to print.

13 Example - Get the list of supported attributes lxpub03:~ > lcg-info --list-attrs Attribute name* Glue object class Glue attribute name EstRespTime GlueCE GlueCEStateEstimatedResponseTime WorstRespTime GlueCE GlueCEStateWorstResponseTime TotalJobs GlueCE GlueCEStateTotalJobs TotalCPUs GlueCE GlueCEInfoTotalCPUs MaxRunningJobs GlueCE GlueCEPolicyMaxRunningJobs CE GlueCE GlueCEUniqueID CEStatus GlueCE GlueCEStateStatus CEVOs GlueCE GlueCEAccessControlBaseRule FreeCPUs GlueCE GlueCEStateFreeCPUs RunningJobs GlueCE GlueCEStateRunningJobs MaxWCTime GlueCE GlueCEPolicyMaxWallClockTime Accesspoint GlueCESEBind GlueCESEBindCEAccesspoint CloseCE GlueCESEBindGroup GlueCESEBindGroupCEUniqueID... GLUE: Grid Laboratory for a Uniform Environment * see "gLite3 user guide"

14 Example - List all the CEs satisfying given conditions and print the desired attributes (How many jobs are running and how many free CPUs there are on CEs that have an Athlon CPU and have Scientific Linux): lxpub03:~ > lcg-info --list-ce --query Processor=*thlon*,OS=*Scientific* -- attrs 'RunningJobs,FreeCPUs‘ - CE: alice003.nipne.ro:2119/jobmanager-lcgpbs-alice - RunningJobs 0 - FreeCPUs 2 - CE: alice003.nipne.ro:2119/jobmanager-lcgpbs-dteam - RunningJobs 0 - FreeCPUs 2 - CE: alice003.nipne.ro:2119/jobmanager-lcgpbs-ops - RunningJobs 0 - FreeCPUs 2 - CE: ce02.lip.pt:2119/jobmanager-lcgsge-atlasgrid - RunningJobs 1 - FreeCPUs 48....

15 The Local GRIS (Grid Resource Information Service) $ ldapsearch -x -h -p -b "mds-vo-name=..., o=grid" where: -x simple authentication (instead of LDAPs SASL) -h hostname -p port (2135 or 2170) -b “mds-vo-name=local(2135)/resource(2170) “ (GRIS level) “mds-vo-name= “ (site level) “mds-vo-name=mds-vo-name=, mds-vo-name=local “ (top-level BDII) ldapsearch -x -h -p 2135 -b "mds-vo-name=local, o=grid" ldapsearch -x -h -p 2170 -b "mds-vo-name=resource,o=grid" where: -x simple authentication (instead of LDAPs SASL) -h hostname -p port (= 2135 or 2170) -b specify the initial entry for the search in the LDAP tree

16 Using the ldapsearch command to read the MDS: ldapsearch -x -H -b "mds-vo-name=local, o=grid" where: ldap_uri = ldap://hostname:port Example - Interrogating the GRIS on a Computing Element ldapsearch -x -h lxb2006.cern.ch -p 2135 -b "mds-vo-name=local, o=grid" or: ldapsearch -x -H ldap://lxb2006.cern.ch:2135 -b "mds-vo-name=local, o=grid"

17 lxpub03:~ > ldapsearch -x -h lcgce01.jinr.ru -p 2135 -b "mds-vo- name=local, o=grid" | less version: 2 # # filter: (objectclass=*) # requesting: ALL # # lcgce01.jinr.ru:2119/jobmanager-lcgpbs-alice, local, grid dn: GlueCEUniqueID=lcgce01.jinr.ru:2119/jobmanager-lcgpbs-alice,mds-vo-name=lo cal,o=grid objectClass: GlueCETop objectClass: GlueCE GlueCEHostingCluster: lcgce01.jinr.ru GlueCEName: alice GlueCEUniqueID: lcgce01.jinr.ru:2119/jobmanager-lcgpbs-alice GlueCEInfoGatekeeperPort: 2119 GlueCEInfoHostName: lcgce01.jinr.ru.....

18 Example -To test site GIIS : lxpub03:~ > ldapsearch -x -H ldap://lcgsite.jinr.ru:2170 \ -b mds-vo-name=JINR-LCG2, o=grid | less # extended LDIF # # LDAPv3 # base with scope sub # filter: (objectclass=*) # requesting: ALL #.... # lxfs07.jinr.ru, lcgce01.jinr.ru:2119/jobmanager-lcgpbs-hone, JINR-LCG2, grid dn: GlueCESEBindSEUniqueID=lxfs07.jinr.ru,GlueCESEBindGroupCEUniqueID= lcgce01.jinr.ru:2119/jobmanager-lcgpbs-hone,Mds-Vo-name=JINR-LCG2,o=grid GlueCESEBindSEUniqueID: lxfs07.jinr.ru GlueCESEBindCEUniqueID: lcgce01.jinr.ru:2119/jobmanager-lcgpbs-hone GlueCESEBindMountInfo: /pnfs/jinr.ru/data # VO-atlas-production-12.0.6, lcgce01.jinr.ru, lcgce01.jinr.ru, JINR-LCG2, grid dn: GlueLocationLocalID=VO-atlas-production-12.0.6,GlueSubClusterUniqueID= lcgce01.jinr.ru,GlueClusterUniqueID=lcgce01.jinr.ru,Mds-Vo-name=JINR-LCG2,o=grid GlueLocationLocalID: VO-atlas-production-12.0.6 GlueLocationName: VO-atlas-production-12.0.6 GlueLocationVersion: Prod GlueLocationPath: $VO_ATLAS_SW_DIR…

19 MONITORING http://goc.grid.sinica.edu.tw/gstat/ https://lcg-sam.cern.ch:8443/sam/sam.py https://goc.gridops.org/site

20 Data Manager System Storage Elements -Types SE -Data Channel Protocols (GridFTP,srm,ftp..) -Storage Resource Manager (SRM) Catalogs -File Catalog -Replica catalog -File Authorization Service -Metadata Catalog File Transfer -File Transfer Service -File Placement Service

21 The Storage Element is the service which allows a user or an application to store data for future retrieval All data in a SE must be considered read-only and therefore can not be changed unless physically removed and replaced. Different VOs might enforce different policies for space quota management. Storage Elements (SE)

22 Types of Storage Elements : -Classic SE: GridFTP server(gsiftp, insecure RFIO daemon) -CASTOR:a tape mass storage system (RFIO) -dCache: It consists of a server and one or more pool nodes (gsidcap) -LCG Disk pool manager: disk pool manager, suitable for small sites (max 10 TB of total space) Data Channel Protocols : GSIFTP (GridFTP) ~ File Transfer FTP-like gsidcap (GSI dCache Access Protocol) ~ I/O Remote file access insecure RFIO (Remote File I/O) ~ I/O Remote file access secure RFIO(gsirfio) ~ I/O Remote file access SRM (Storage Resource Manager ) : -is a service interacting with local SE and providing the GRID-interface for the external world. - has been designed to be the single interface for the management of disk and tape storage resources. - Srm-protocol is a storage management protocol and not a file access one.

23 File Names in gLite 3.1 *Grid Unique IDentifier (GUID): guid: guid:38ed3f60-c402-11d7-a6b0-f53ee5a37e1d *Logical File Name (LFN) or User Alias: lfn: lfn:importantResults/Test1240.dat, lfn:/grid/ / / *Storage URL (SURL)= Physical File Name (PFN) :// / where: sfn - files located in SEs without a SRM interface srm - SRM-managed SEs sfn:// srm://srm.cern.ch/castor/cern.ch/grid/dteam/doe/file1 *Transport URL (TURL) :// gsiftp://tbed0101.cern.ch/data/dteam/doe/file1 where : = (GSIFTP, RFIO, gsidcap), supported by the SE to access the file

24 File Catalogue in gLite 3.1 File Catalogue is the service which maintains mappings between LFN(s), GUID, SURL(s). Grid file it is both physically present in a SE and registered in the File Catalogue LFC. Client tools : lfc-* commands for LFC catalogue define the match between LFN (Logical File Name), SURL(Site URL) lcg-* commands to upload/download files to/from Grid (SE) replicate data and interact with the file catalogues.

25 LFC Commands lfc-* structure of the LFC : /grid/ / Environment for lfc-* LCG_CATALOG_TYPE=lfc LCG_LOCATION=/opt/lcg LCG_TMP=/tmp LCG_GFAL_INFOSYS=lcgbdii.jinr.ru:2170 LFC_HOST= "name of the LFC server“ LFC_HOME=/grid/atlas/MYDIR lxpub03:~ > lcg-infosites --vo atlas lfc prod-lfc-atlas-central.cern.ch lxpub03:~ > lcg-infosites --vo atlas lfcLocal | grep jinr lcglfc01.jinr.ru lxpub03:~ > export LFC_HOST=lcglfc01.jinr.ru lxpub03:~ > export LFC_CATALOG_TYPE=lfc

26 LFC Commands lfc-* lfc-chmod Change access mode of a LFC file/directory lfc-chown Change owner and group of a LFC file/directory lfc-delcomment Delete the comment associated with a file/directory lfc-getacl Get file/directory access control lists lfc-ln Make a symbolic link to a file/directory lfc-ls List file/directory entries in a directory lfc-mkdir Create directory lfc-rename Rename a file/directory lfc-rm Remove a file/directory lfc-setacl Set file/directory access control lists lfc-setcomment Add/replace a comment lfc-entergrpmap Defines a new group entry in the Virtua table lfc-enterusrmap Defines a new user entry in Virtual ID table lfc-modifygrpmap Modifies a group entry corresponding to a given virtual gid lfc-modifyusrmap Modifies a user entry corresponding to a given virtual uid lfc-rmgrpmap Suppresses group entry corresponding to a given virtual gid or group name lfc-rmusrmap Suppresses user entry corresponding to a given virtual uid or user name.

27 Examples : *Listing the entries of a LFC directory : lxpub03:~ > lfc-ls /grid/ alice atlas biomed cms dteam fusion hone lhcb ops gstest *Creating directories in the LFC : lxpub03:~ > lfc-mkdir /grid/atlas/tutorial lxpub03:~ > lfc-ls -l /grid/atlas/ drwxrwxr-x 0 103 112 0 Apr 10 11:43 SAM drwxrwxr-x 0 107 102 0 Apr 10 21:25 tutorial

28 Examples : *Creation of symbolic links lxpub03:~ > lfc-ln -s /grid/atlas/tutorial /grid/atlas/example lxpub03:~ > lfc-ls -l /grid/atlas/ drwxrwxr-x 0 103 112 0 Apr 10 11:43 SAM lrwxrwxrwx 1 107 102 0 Apr 10 21:27 example -> /grid/atlas/tutorial drwxrwxr-x 0 107 102 0 Apr 10 21:25 tutorial *Adding metadata information to LFC entries lxpub03:~ > lfc-setcomment /grid/atlas/example "Simple Test“ lxpub03:~ > lfc-ls --comment /grid/atlas/example grid/atlas/example Simple Test *Removing LFNs from the LFC lxpub03:~ > lfc-rm /grid/atlas/example

29 Examples : *Access Control Lists : * a user (user); * a group of users (group); * any other user (other); * the maximum permissions granted to specific users or groups (mask). permissions: read (r), write (w), execute (x) *Print the ACL of a directory lxpub03:~ > lfc-getacl /grid/atlas/SAM # file: /grid/atlas/SAM # owner: /C=IT/O=INFN/OU=Personal Certificate/L=Roma 1/CN=Alessandro Di Girolamo # group: atlas/Role=lcgadmin user::rwx group::rwx #effective:rwx other::r-x default:user::rwx default:group::rwx default:other::r-x

30 File and Replica Management Client Tools LCG Data Management Client Tools The LCG Data Management tools lcg_util allow users: to copy files between UI,CE,WN,SE; to register entries in the file catalogue and replicate files between SEs. *Replica Management (FTS) lcg-cp Copies a Grid file to a local destination (download) lcg-cr Copies a file to a SE and registers the file in the catalogue (upload) lcg-del Deletes one file (either one replica or all replicas) lcg-rep Copies a file from one SE to another SE and registers it in the catalogue (replicate) lcg-gt Gets the TURL for a given SURL and transfer protocol lcg-sd Sets file status to "Done" for a given SURL in an SRM's request

31 *File Catalogue Interaction (FTS LFS) lcg-aa Adds an alias in the catalogue for a given GUID lcg-ra Removes an alias in the catalogue for a given GUID lcg-rf Registers in the catalogue a file residing on an SE lcg-uf Unregisters in the the catalogue a file residing on an SE lcg-la Lists the aliases for a given LFN, GUID or SURL lcg-lg Gets the GUID for a given LFN or SURL lcg-lr Lists the replicas for a given LFN, GUID or SURL

32 Environment for lcg-* LCG_GFAL_INFOSYS= lcgbdii.jinr.ru:2170 LFC_HOST = lcglfc01.jinr.ru LFC_HOME = /grid/atlas/tutorial LCG_GFAL_VO =atlas VO_ATLAS_DEFAULT_SE= lxfs07.jinr.ru lxpub03:~ > echo $LFC_HOST lcglfc01.jinr.ru lxpub03:~ > echo $LCG_GFAL_INFOSYS lcgbdii.jinr.ru:2170 lxpub03:~ > echo $VO_ATLAS_DEFAULT_SE lxfs07.jinr.ru

33 Examples *Uploading a file to the Grid Syntax: lcg-cr [--config config_file] [-d dest_file | dest_host] [-g guid] [-h |--help] [-i | --insecure] [-l lfn] [-n nbstreams] [-P relative_path] [-t timeout] [-v |--verbose] [--vo vo] src_file where: dest_file= SURL (sfn://, srm://) -d lxb0710.cern.ch -d sfn://lxb0710.cern.ch/data/dteam/my_file -d lxb0710.cern.ch -P my_dir/my_file dest_host = SE guid = GUID file lfn = LFN vo = VO src_file = file:///, gsiftp:///

34 lxpub04:~/data > lcg-cr -v --vo atlas -d lxdpm104.cern.ch -l lfn:/grid/atlas/tutorial/File2.txt file:///afs/jinr.ru/user/g/grom/data/testFile.txt Using grid catalog type: lfc Using grid catalog : lcglfc01.jinr.ru Using LFN : /grid/atlas/tutorial/File2.txt Using SURL : srm://lxdpm104.cern.ch/dpm/cern.ch/home/atlas/generated/2008-04- 14/file8232e17f-6186-4f73-95b3-dc3f6d496fd1 Source URL: file:///afs/jinr.ru/user/g/grom/data/testFile.txtFile size: 24 VO name: atlas Destination specified: lxdpm104.cern.ch Destination URL for copy: gsiftp://lxdpm104.cern.ch/lxdpm104.cern.ch:/data01/atlas/2008-04-14/file8232e17f- 6186-4f73-95b3-dc3f6d496fd1.141352.0 # streams: 1 # set timeout to 0 seconds Alias registered in Catalog: lfn:/grid/atlas/tutorial/File2.txt 24 bytes 0.01 KB/sec avg 0.01 KB/sec inst Transfer took 2040 ms Destination URL registered in Catalog: srm://lxdpm104.cern.ch/dpm/cern.ch/home/atlas/generated/2008-04- 14/file8232e17f-6186-4f73-95b3-dc3f6d496fd1guid:da781a56-5450-470e-9ae9- 7e2950dd1d1f guid:da781a56-5450-470e-9ae9-7e2950dd1d1f

35 lxpub04:~/data > lfc-ls -l /grid/atlas/tutorial -rw-rw-r-- 1 107 102 24 Apr 14 16:48 File2.txt -rw-rw-r-- 1 107 102 5 Apr 14 16:54 new -rw-rw-r-- 1 107 102 19896 Apr 14 16:42 test

36 *Copies a file from one SE to another SE and registers it in the catalogue (replicate) Once a file is stored on an SE and registered in the catalogue, the file can be replicated using the lcg-rep command Syntax: lcg-rep [--config config_file] [-d dest_file | dest_host] [-h | --help] [-i | --insecure] [-n nbstreams] [-P relative_path] [-t timeout] [-v | --verbose] [--vo vo] src_file where: -d dest_file = SURL file (sfn://,srm://) -dest_host = SE src_file = source file : LFN/GUID/SURL lxpub04:~/data > lcg-infosites --vo atlas se | grep cern 1000000000000 500000000000 n.a castorsrm.cern.ch 236930000 34904 n.a lxdpm104.cern.ch 300000000000 160000000000 n.a srm-atlas.cern.ch 1000000000000 500000000000 n.a srm.cern.ch....

37 lxpub04:~/data >lcg-rep -v --vo atlas -d srm-atlas.cern.ch guid:da781a56- 5450-470e-9ae9-7e2950dd1d1f Using grid catalog type: lfc Using grid catalog : lcglfc01.jinr.ru srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/generated/2008-04- 14/file8267edf1-60e8-46b7-a16b-bcfee2b16484: Source URL: guid:da781a56-5450-470e-9ae9-7e2950dd1d1f File size: 24 VO name: atlas Destination specified: srm-atlas.cern.ch Source URL for copy: gsiftp://lxdpm104.cern.ch/lxdpm104.cern.ch:/data01/atlas/2008-04-14/file8232e17f- 6186-4f73-95b3-dc3f6d496fd1.141352.0 Destination URL for copy: gsiftp://lxfsrd1004.cern.ch:2811//castor/cern.ch/grid/atlas/generated/2008-04- 14/file8267edf1-60e8-46b7-a16b-bcfee2b16484 # streams: 1 # set timeout to 0 24 bytes 0.03 KB/sec avg 0.03 KB/sec inst Transfer took 15140 ms srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/generated/ 2008-04-14/file8267edf1-60e8-46b7-a16b-bcfee2b16484: Destination URL registered in LRC: srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/generated/2008-04- 14/file8267edf1-60e8-46b7-a16b-bcfee2b16484

38 *Listing replicas, GUIDs and aliases The lcg-lr command allows users to list all the replicas of a file registered in the file catalogue: Syntax: lcg-lr [--config config_file] [-h | --help] [-i | --insecure] [--vo vo] file lxpub04:~/data > lcg-lr --vo atlas lfn:/grid/atlas/tutorial/File2.txt srm://lxdpm104.cern.ch/dpm/cern.ch/home/atlas/generated/2008-04- 14/file8232e17f-6186-4f73-95b3-dc3f6d496fd1 srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/generated/2008-04- 14/file8267edf1-60e8-46b7-a16b-bcfee2b16484

39 *Copies a Grid file to a local destination (download) Syntax: lcg-cp [-h | --help] [-i | --insecure] [--config config_file] [-n nbstreams] [-t timeout] [-v | --verbose] [--vo vo] src_file dest_file lxpub0:~/data > lcg-cp -v --vo atlas lfn:testFile.txt file:///$PWD/testFile_from-grid.txt Using grid catalog type: lfc Using grid catalog : lcglfc01.jinr.ru VO name: atlas srm://srm-atlas.cern.ch/castor/cern.ch/grid/atlas/generated/2008-04- 14/file8267edf1-60e8-46b7-a16b-bcfee2b1 Source URL: lfn:/grid/atlas/tutorial/File2.txt File size: 24 Source URL for copy: gsiftp://lxfsrd1004.cern.ch:2811//castor/cern.ch/grid/atlas/generated/2008-04- 14/file8267edf1-60e8-46b7-a16b-bcfee2b16484 Destination URL: file:/afs/jinr.ru/user/g/grom/data/File2_from-Grid.txt # streams: 1 # set timeout to 0 (seconds) 0 bytes 0.00 KB/sec avg 0.00 KB/sec inst Transfer took 4010 ms testFile_from.txt

40 lxpub04:~/data > ls -l total 691 -rw-r--r-- 1 grom lcta 5 Apr 14 16:52 new -rw-r--r-- 1 grom lcta 24 Apr 12 2007 testFile.txt -rw-r--r-- 1 grom lcta 24 Apr 13 2007 testFile_from.txt

41 *Gets the GUID for a given LFN or SURL Syntax: lcg-lg [--config config_file] [-h | --help] [-i | --insecure] [--vo vo] lfn_or_surl lxpub04:~/data > lcg-lg --vo atlas lfn:/grid/atlas/tutorial/File2.txt guid:da781a56-5450-470e-9ae9-7e2950dd1d1f

42 *Gets the TURL for a given SURL and transfer protocol Syntax: lcg-gt [-t timeout] surl protocol lxpub04:~/data > lcg-gt srm://lxdpm104.cern.ch/dpm/cern.ch/home/atlas/ generated/2008-04-14/file8232e17f-6186-4f73-95b3-dc3f6d496fd1 gsiftp gsiftp://lxdpm104.cern.ch/lxdpm104.cern.ch:/data01/atlas/2008-04-14/file8232e17f- 6186-4f73-95b3-dc3f6d496fd1.141352.0141380

43 *Adds an alias in the catalogue for a given GUID Syntax: lcg-aa [--config config_file] [-h | --help] [-i | --insecure] [--vo vo] guid lfn lxpub04:~/data > lcg-aa --vo atlas da781a56-5450-470e-9ae9-7e2950dd1d1f lfn:/grid/atlas/tutorial/Dubl_File2 lxpub04:~/data > lfc-ls -l /grid/atlas/tutorial/ lrwxrwxrwx 1 107 102 0 Apr 14 17:40 Dubl_File2 -> /grid/atlas/tutorial/File2.txt -rw-rw-r-- 1 107 102 24 Apr 14 16:48 File2.txt -rw-rw-r-- 1 107 102 5 Apr 14 16:54 new -rw-rw-r-- 1 107 102 19896 Apr 14 16:42 test

44 *Lists the aliases for a given LFN, GUID or SURL Syntax: lcg-la [--config config_file] [-h | --help] [-i | --insecure] [--vo vo] file lxpub04:~/data > lcg-la --vo atlas lfn:/grid/atlas/tutorial/File2.txt lfn:/grid/atlas/tutorial/File2.txt lfn:/grid/atlas/tutorial/Dubl_File2

45 *Deletes one file (either one replica or all replicas) Syntax: lcg-del [-a] [--config config_file] [-h | --help] [-i | --insecure] [-t timeout] [-s se] [-v | --verbose] [--vo vo] file lxpub04:~/data > lfc-ls -l /grid/atlas/tutorial lrwxrwxrwx 1 107 102 0 Apr 14 17:40 Dubl_File2 -> /grid/atlas/tutorial/File2.txt -rw-rw-r-- 1 107 102 24 Apr 14 16:48 File2.txt -rw-rw-r-- 1 107 102 5 Apr 14 16:54 new -rw-rw-r-- 1 107 102 19896 Apr 14 16:42 test lxpub04:~/data > lfc-getacl /grid/atlas/tutorial # file: /grid/atlas/tutorial # owner: /C=RU/O=RDIG/OU=users/OU=jinr.ru/CN=Natalia Gromova # group: atlas user::rwx group::rwx #effective:rwx other::r-x default:user::rwx default:group::rwx default:other::r-x

46 lxpub04:~/data > lcg-del -v -a --vo atlas lfn:test VO name: atlas Using GUID : 7e7795b5-6838-4e58-93af-e56238867aab set timeout to 0 seconds srm://lxdpm104.cern.ch/dpm/cern.ch/home/atlas/generated/2008-04-14/ file37070a30-9e01-42a2-9ef6-8d2806056a03 is deleted srm://lxdpm104.cern.ch/dpm/cern.ch/home/atlas/generated/2008-04-14/ file37070a30-9e01-42a2-9ef6-8d2806056a03 is unregistered lxpub04:~/data > lfc-ls -l /grid/atlas/tutorial lrwxrwxrwx 1 107 102 0 Apr 14 17:40 Dubl_File2 -> /grid/atlas/tutorial/File2.txt -rw-rw-r-- 1 107 102 24 Apr 14 16:48 File2.txt -rw-rw-r-- 1 107 102 5 Apr 14 16:54 new


Download ppt "БАЗОВЫЕ СРЕДСТВА РАБОТЫ С ФАЙЛАМИ В GRID. The gLite3 Architecture Security Service: Grid Security Infrastructure (GSI) Secure Sockets Layer (SSL) communication."

Similar presentations


Ads by Google