Presentation is loading. Please wait.

Presentation is loading. Please wait.

Introduction to iRODS Christine Staiger SURFsara Amsterdam, 8 th May 2014 10 th Oct 2014.

Similar presentations

Presentation on theme: "Introduction to iRODS Christine Staiger SURFsara Amsterdam, 8 th May 2014 10 th Oct 2014."— Presentation transcript:

1 Introduction to iRODS Christine Staiger SURFsara Amsterdam, 8 th May 2014 10 th Oct 2014

2 iRODS Concepts The Basics ① Installation ② Tools for administration ③ Adding data and metadata Contents

3 iRODS Concepts 3

4 What is iRODS? 4

5 What does iRODS provide? Storage virtualization of different disk and tape storage systems A logical namespace across storage locations A rule engine to automate data management according to defined policies Various client interfaces 5

6 iRODS Components 6

7 iRODS Rule Engine 7

8 The users view: iRODS Data Collections 8 Collection 0 Data Object Data Object Data Object Data Object A Collection 1 Data Object Data Object Data Object X …/Collection0/ …/Collection0/DataObject A-B …/Collection0/Collection1/ …/Collection0/Collection1/DataObject X-Z Logical Name Space (iCAT) independent from physical resources Similar to Unix Directory and Files ichmod to set user/group ACLs msiSetDefaultResc(Resource).irodsEnv: irodsDefResource=Resource Similar to Unix Directory and Files ichmod to set user/group ACLs msiSetDefaultResc(Resource).irodsEnv: irodsDefResource=Resource

9 In the background: iRODS Resources (Storage) Resource is a Software or Hardware system that stores data 3 Resource classes: 9 High Latency High Latency Low Latency Low Latency Archive Resource Archive Resource Cache Resource Cache Resource Compound Resource Compound Resource POSIX like IO calls

10 iRODS Resource Groups 10 Low Latency Low Latency Cache Resource Cache Resource POSIX like IO calls Resource Group Resource Group Low Latency Low Latency Cache Resource Cache Resource Low Latency Low Latency Cache Resource Cache Resource @Server A@Server B@Server C

11 Hands on The Basics 11

12 Demo Data Grids Virtual machines: ssh user@IP IP adressUserPassword

13 Installation and configuration Execute./irodssetup in the folder iRODS Export the icommands export PATH=$PATH:/home/user/iRODS/clients/icommands/bin ~/iRODS/config/irods.config # Database configuration $DATABASE_TYPE = 'postgres'; … $DATABASE_HOST = 'localhost'; $DATABASE_PORT = '5432'; $DATABASE_ADMIN_PASSWORD = 'irods2'; $DATABASE_ADMIN_NAME = ’alice'; # iRODS configuration IRODS_HOME = '/home/alice/iRODS'; $IRODS_PORT = '1247’; … $IRODS_ADMIN_PASSWORD = ’alice'; $IRODS_ICAT_HOST = '';

14 .irodsEnv file # iRODS personal configuration file. # # This file was automatically created during iRODS installation. # Created Mon Jun 18 13:13:27 2012 # # iRODS server host name: irodsHost ’surfsara-cloud' # iRODS server port number: irodsPort 1247 # Default storage resource name: irodsDefResource 'demoResc' # Home directory in iRODS: irodsHome '/alicesZone/home/alice' # Current directory in iRODS: irodsCwd '/alicesZone/home/alice' # Account name: irodsUserName ’alice' # Zone: irodsZone ‘alicesZone' The.irodsEnv determines the zone and the user for the icommands-client. You can only use one such file at a time.

15 iRODS i-commands Documentation: (we don’t provide a exhaustive list of i-commands)

16 iRODS control Execute./irodsctl in the folder iRODS Restarting, starting and stopping the iRODS and iCAT server with = restart, start, stop Restarting, starting and stopping only the iRODS server with = irestart, istart, istop Initialise an account and log in with iinit 16

17 The icommands client icommands are automatically installed when you install the iRODS server Users must install the client software to connect to an iRODS server Categories: Informational UNIX and FTP like Metadata Functional

18 ihelp Lists all commands ihelp -h Usage : ihelp [-ah] [icommand] Display i-commands synopsis or a particular i-command help text Options are: -h this help -a print the help text for all the i-commands Run with no options to display a synopsis of the i-commands iRODS Version 3.3.1 February 2014 ihelp

19 ienv Lists the iRODS environment variables. Equivalent to “iinit -l” Usage : ienv [-h] NOTICE: Release Version = rods3.3.1, API Version = d NOTICE: irodsHost=surfsara-cloud NOTICE: irodsPort=1247 NOTICE: irodsDefResource=demoResc NOTICE: irodsHome=/alicesZone/home/alice NOTICE: irodsCwd=/alicesZone/home/alice NOTICE: irodsUserName=alice NOTICE: irodsZone=alicesZone

20 iuserinfo Show information about your iRODS user account or the entered user Usage: iuserinfo [-vVh] [user] name: alice id: 10007 type: rodsadmin zone: alicesZone info: comment: create time: 01396887567: 2014-04-07.18:19:27 modify time: 01396887567: 2014-04-07.18:19:27 member of group: alice member of group: public

21 ilsresc ilsresc lists iRODS resources and resource-groups Usage: ilsresc [-lvVhA] [Name] alice@sara-cloud:~$ ilsresc demoResc alice@sara-cloud:~$ ilsresc –l demoResc resource name: demoResc resc id: 10010 zone: alicesZone type: unix file system class: archive location: surfsara-cloud vault: /home/alice/iRODS/Vault …

22 Some commands Unix-like: ils, ipwd, icd, imkdir, irm, icp ichmod, ipasswd, irsync, ichksum, imv FTP-like: iinit, iput, iget, iexit Test: Create folder in the logical name space ipwd imkdir NewFolder ils NewFolder irm –r NewFolder

23 User and Data administration Change to admin mode: iadmin Quit admin mode: q Make/delete a user: mkuser, rmuser, moduser (iadmin) mkuser user1 rodsuser (iadmin) moduser user1 password ***** Parameters and their values: iadmin lt, iadmin lt user_type iadmin mkresc, iadmin rmresc Create new resource: mkdir Data (in your home, not in the iRODS environment) iadmin mkresc NewDataResc “unix file system” cache “/home/alice/Data”

24 Put and replicate data Add the file /home/login/put1.txt to resource iput -v put1.txt ils -l or ils -L iput -h ichksum put1.txt (checksum of copy in Vault) md5sum put1.txt (checksum of file in /home/login) iput -R NewDataResc put2.txt ils -L Replicate data: irepl -R NewDataResc put1.txt Trim number of copies: itrim -S demoResc -N 1 put1.txt

25 Access Control Lists Change permission for a user: ichmod read user1 put1.txt List access: ils -A Change permissions for group “public”: Make a directory in the iRODS environment: imkdir temp Make directory readable for “public”: ichmod -r read public temp ils -A /bobsZone/home/bob/temp: ACL - bob#bobsZone:own g:public#bobsZone:read object Inheritance - Disabled

26 iRODS Metadata Commands: imeta, iquest, idbo Add metadata: imeta add -d put1.txt “Date” “08.10.2013” imeta add -d put1.txt “AnotherDate” “04.04.2014” imeta ls -d put1.txt Remove metadata: imeta rm -d put1.txt “Date” “08.10.2013” Querying the iCAT: iquest "SELECT DATA_NAME,DATA_CHECKSUM WHERE DATA_RESC_NAME like ’%Resc%’” DATA_NAME = put1.txt DATA_CHECKSUM = c16d7d3488677a3348b12eb82795d28c

27 Federation between data grids List all existing zones: ils / Zone A acknowledges Zone B and vice-cersa iadmin mkzone B remote Host:Port Zone A adds remote user from Zone B: iadmin mkuser user#B rodsuser Example: Alice and Bob: Alice acknowledges Bob’s zone: iadmin mkzone bobsZone remote Bob acknowledges Alice’s zone: iadmin mkzone alicesZone remote Alice creates remote user Bob: iadmin mkuser bob#bobsZone rodsuser Now Bob can list and alter files in Alice’s zone: ils /alicesZone

28 Data Replication between grids Bob copies a file from his zone to his account in Alice’s zone irsync -R demoResc i:/bobsZone/home/bob/put1.txt i:/alicesZone/home/bob#bobsZone/put1.txt -R specifies the target resource Bob copies a file from his home directory to Alice’s zone irsync -R demoResc put3.txt i: /alicesZone/home/bob#bobsZone/put3.txt Bob copies a file from Alice’s zone to his home directory mkdir test irsync i:/alicesZone/home/bob#bobsZone/put3.txt test/put3.txt Bob deletes a file in Alice’s zone irm /alicesZone/home/bob#bobsZone/put3.txt

29 Thank you !

30 Appendix

31 Trouble shooting ERROR: putUtil: put error for /alicesZone/home/bob#bobsZone/put1.txt, status = -9000 status = -9000 SYS EXCEED CONNECT CNT Host configuration problem on the server. Solution: Edit iRODS/server/config/irodsHost localhost you.rIP.XX.XX surfsara-cloud localhost restart

Download ppt "Introduction to iRODS Christine Staiger SURFsara Amsterdam, 8 th May 2014 10 th Oct 2014."

Similar presentations

Ads by Google