1. LFC Installation and Configuration
Dong Xu
IHEP, CAS
Asia EPIKH School for Grid Site Administrators
Beijing,

2. Outline Introduction to LFC Installation and configuration tutorial
Algiers, Joint EPiKH/EUMEDGRID-Support in Algeria,

3. Introduction LFC: The LCG File Catalog
Users and programs produce and require data
Data may be stored in Grid datasets (files)
Located in Storage Elements (SEs)
Several replicas of one file in different sites
Accessible by Grid users and applications from “anywhere”
Locatable by the WMS (data requirements in JDL)
Data may be copied from/to local filesystems (WNs, UIs) to the Grid
Why File Catalogs are used?
They keep track of the locations of copies (replicas) of Grid files
They provide DM tools and APIs through which users and grid services can deal with files

4. SE Types Disk pool managers (dCache and gLite DPM) Classic SE:
manage distributed storage servers in a centralized way
Physical disks or arrays are combined into a common (virtual) file system
Disks can be dynamically added to the pool
GridFTP server
Secure remote access protocols (gsidcap for dCache, gsirfio for DPM)
SRM interface
Classic SE:
Insecure RFIO daemon (rfiod) – only LAN limited file access
Single disk or disk array
No quota management
Does not support the SRM interface
Mass Storage Systems (Castor- CERN Advanced STORage manager)
Files migrated between front-end disk and back-end tape storage hierarchies
Insecure RFIO (Castor)
Provide a SRM interface with all the benefits

5. SRM (Storage Resource Manager )
dCache
Own system, own protocols
and parameters
You as a user need to know all the systems!!!
I talk to them on your behalf
I will even allocate space for your files
And I will use transfer protocols to send your files there
gLite DPM
Independent system from
dCache or Castor
SRM
Castor
No connection with
dCache or DPM

6. SRM Interactions SRM Client Storage 4 1 2 3 5
The client asks the SRM for the file providing an SURL (Site URL)
The SRM asks the storage system to provide the file
The storage system notifies the availability of the file and its location
The SRM returns a TURL (Transfer URL), i.e. the location from where the file can be accessed
The client interacts with the storage using the protocol specified in the TURL

7. Files Naming Conventions
Logical File Name (LFN)
An alias created by a user to refer to some item of data,
e.g. “lfn:/grid/gilda/ /run2/track1”
Globally Unique Identifier (GUID)
A non-human-readable unique identifier for an item of data, e.g.
“guid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6”
Site URL (SURL) (or Physical File Name (PFN) or Site FN)
The location of an actual piece of data on a storage system,
e.g. “srm://grid009.ct.infn.it/dpm/ct.infn.it/gilda/output10_1” (SRM)
Transport URL (TURL)
Temporary locator of a replica + access protocol: understood by a SE,
e.g. “rfio://lxshare0209.cern.ch//data/alice/ntuples.dat”

8. What is a file catalog?
File Catalog SE SE SE
gLite UI

9. What is a File Catalog? Each file has a unique identifier
Files/directories are organized on a Catalogue
Similar to a filesystem (Logical File Name)
There is one Catalogue per VO
The data can be stored on several Storage Elements (SE)
The Catalogue hides the actual location
Logical File Name
LFN : /grid/gilda/dornbirn/file.txt
Storage Resource Manager
srm://trigrid-ce01.unime.it/dpm/unime.it/home/gilda/generated/ /filef026441a f-b28d-06cb7e4c784f
Physical Filename
/home/gilda/generated/ /filef026441a f-b28d-06cb7e4c784f
Catalogue SE SE SE SE SE

10. Different Types of Catalog
File Catalog
Filesystem-like view on logical file names
Keeps track of sites where data is stored
Conflict resolution
Replica Catalog
Keeps information at a site
(Meta Data Catalog)
Attributes of files on the logical level
Boundary between generic middleware and application layer
Metadata
Catalog
Metadata
Catalog
Metadata
Catalog
Metadata
Metadata
Metadata
File Catalog
LFN
GUID
Site ID
File Catalog
File Catalog
LFN
GUID
Site ID
Site ID
Site ID
Replica Catalog Site A
GUID
SURL
LFN
Replica Catalog Site A
Replica Catalog Site B
GUID
SURL
LFN
Replica Catalog Site B
Replica Catalog Site B
LFN
GUID
SURL
LFN
GUID
SURL
SURL
SURL

11. LFC Commands Summary of the LFC Catalog commands lfc-chmod
Change access mode of the LFC file/directory
lfc-chown
Change owner and group of the LFC file-directory
lfc-delcomment
Delete the comment associated with the file/directory
lfc-getacl
Get file/directory access control lists
lfc-ln
Make a symbolic link to a file/directory
lfc-ls
List file/directory entries in a directory
lfc-mkdir
Create a directory
lfc-rename
Rename a file/directory
lfc-rm
Remove a file/directory
lfc-setacl
Set file/directory access control lists
lfc-setcomment
Add/replace a comment

12. lcg utils commands Replica Management File Catalog Interaction lcg-aa
lcg-cp
Copies a grid file to a local destination
lcg-cr
Copies a file to a SE and registers the file in the catalog
lcg-del
Delete one file
lcg-rep
Replication between SEs and registration of the replica
lcg-gt
Gets the TURL for a given SURL and transfer protocol
lcg-sd
Sets file status to “Done” for a given SURL in a SRM request
File Catalog Interaction
lcg-aa
Add an alias in LFC for a given GUID
lcg-ra
Remove an alias in LFC for a given GUID
lcg-rf
Registers in LFC a file placed in a SE
lcg-uf
Unregisters in LFC a file placed in a SE
lcg-la
Lists the alias for a given SURL, GUID or LFN
lcg-lg
Get the GUID for a given LFN or SURL
lcg-lr
Lists the replicas for a given GUID, SURL or LFN

13. Installation and configuration

14. Check your host Vi /etc/hosts (for example) Vi /etc/hosts.allow
# Do not remove the following line, or various programs
# that require network functionality will fail.
localhost.localdomain localhost
gilda01.ihep.ac.cn gilda01
Vi /etc/hosts.allow
~]# more /etc/hosts.allow #
# hosts.allow This file describes the names of the hosts which are
# allowed to use the local INET services, as decided
# by the '/usr/sbin/tcpd' server.
slapd: all
Start ntpd
ntpdate ntp-1.infn.it
service ntpd start

15. Repository settings Specify the mrepo host:
export MREPO=
Configure the repository as follows:
REPOS="dag ig glite-generic lcg-ca gilda glite-lfc_mysql”
Get repositories with:
for name in $REPOS;
do wget $MREPO/$name.repo -O /etc/yum.repos.d/$name.repo; done
Copy the gilda utils:
wget -O /etc/yum.repos.d/gilda.repo

16. install by yum update the host: Update java yum install java
yum clean all
Update java
yum install java
generic installation:
update CA:
yum install lcg-CA
install Gilda utils:
yum install gilda_utils
UI specific Install:
install LFC:
yum install ig_LFC_mysql

17. get host certificates ready
$# cp hostcert.pem /etc/grid-security/
$# cp hostkey.pem /etc/grid-security/
$# chmod 644 /etc/grid-security/hostcert.pm
$# chmod 400 /etc/grid-security/hostkey.pem

18. edit /opt/glite/yaim/examples/siteinfo/ig-site-info.def
SITE_NAME=gilda-BJ-IHEP
SITE_ ="your address"
MYSQL_PASSWORD=123456
GROUPS_CONF=/opt/glite/yaim/examples/ig-groups.conf
USERS_CONF=/opt/glite/yaim/examples/ig-users.conf
JAVA_LOCATION=/usr/lib/jvm/java openjdk x86_64/jre
VOS="gilda"
VO_GILDA_SW_DIR=$VO_SW_DIR/gilda
VO_GILDA_DEFAULT_SE=$DPM_HOST
VO_GILDA_STORAGE_DIR=$CLASSIC_STORAGE_DIR//gilda
VO_GILDA_QUEUES="short long infinite"
VO_GILDA_VOMS_SERVERS="vomss://voms.ct.infn.it:8443/voms/gilda?/gilda"
VO_GILDA_VOMSES="'gilda voms.ct.infn.it /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it gilda'"
VO_GILDA_VOMS_CA_DN="'/C=IT/O=INFN/CN=INFN CA' '/C=IT/O=INFN/CN=INFN CA'“
LFC_HOST=gilda17.ihep.ac.cn # add a new line
LFC_DB=cns_db_lfc # add a new line
LFC_DB_HOST=$LFC_HOST # add a new line
LFC_DB_PASSWORD= # add a new line
LFC_HOST_ALIAS="" # add a new line
LFC_CENTRAL="gilda" # add a new line
LFC_LOCAL="" # add a new line
LFC_READONLY=no # add a new line

19. Edit ig-groups.conf
Edit /opt/glite/yaim/examples/siteinfo/ig-groups.conf,
"/gilda/ROLE=SoftwareManager":::sgm:
"/gilda/grelc/das/*":gilda:::
"/gilda"::::
Edit /etc/hosts.allow
sldap: all # add a new line

20. Edit other files edit ig-lfc_mysql
vi /opt/glite/yaim/examples/siteinfo/services/ig-lfc_mysql
LFC_HOST=gilda03.ihep.ac.cn #replace your hostname
LFC_DB_PASSWORD=123456

21. configure LFC with ig-yaim
$# /opt/glite/yaim/bin/ig_yaim -c -s /opt/glite/yaim/examples/siteinfo/ig-site-info.def -n ig_LFC_mysql

22. test the function of LFC
test the function of LFC(from UI with valid user credential)
$# export LFC_HOST=your_lfc_host
$# lfc-ls /grid/gilda
$# lfc-mkdir /grid/gilda/home
$# lfc-mkdir /grid/gilda/home/xud
$# lcg-cr --vo gilda -l lfn:/grid/gilda/home/xud/testfile3 -d gilda-02.pd.infn.it file:////etc/hosts
$# lfc-ls -l /grid/gilda/home/wuwj/testfile3

23. [xud@gilda08 .globus]$ voms-proxy-init -voms gilda
Enter GRID pass phrase:
Your identity: /C=CN/O=HEP/O=IHEP/OU=CC/CN=dong xu
Creating temporary proxy Done
Contacting voms.ct.infn.it:15001 [/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it] "gilda" Done
Creating proxy Done
Your proxy is valid until Tue Aug 31 09:36:
.globus]$ echo $LFC_HOST
gilda07.ihep.ac.cn

24. [xud@gilda08 .globus]$ lfc-ls /grid/gilda
generated
malanxin
mlx
tutorials
.globus]$ lfc-mkdir /grid/gilda/home
home
.globus]$ lfc-mkdir /grid/gilda/home/xud
.globus]$ lcg-cr --vo gilda -l lfn:/grid/gilda/home/xud/testfile3 -d gilda-02.pd.infn.it file:////etc/hosts
guid:48943c1e-76b a2c8-5437d3d7bdbc
.globus]$ lfc-ls -l /grid/gilda/home/xud/testfile3
-rw-rw-r Aug 30 21:38 /grid/gilda/home/xud/testfile3

25. Bibliography Information on the file catalogs LFC, gfal, lcgutils:
“Evolution of LCG2 Data Management (JP Baud, J. Casey)”
LFC installation, administration, migration from RLS:
Wiki entries indicated through the presentation:

26. Thank you very much!