1. gLite SE(DPM) Installation
ZHU Wei
CC-IHEP
1 September 2010

2. Outline
OVERVIEW
INSTALLATION & CONFIGURATION
TESTING

3. OVERVIEW The Grid DM Challenge Heterogeneity
Data are stored on different storage systems using different access technologies
Distribution
Data are stored in different locations – in most cases there is no shared file system or common namespace
Data need to be moved between different locations
Need common interface to storage resources
Storage Resource Manager (SRM)
Need to keep track where data is stored
File and Replica Catalogs
Need scheduled, reliable file transfer
File transfer service
Heterogeneity 不同成分

4. OVERVIEW Assumptions: Files: Also…
Users and programs produce and require data
the lowest granularity of the data is on the file level (we deal with files rather than data objects or tables)
Data = files
Files:
Mostly, write once, read many
Located in Storage Elements (SEs)
Several replicas of one file in different sites
Accessible by Grid users and applications from “anywhere”
Also…
WMS can send (small amounts of) data to/from jobs: Input and Output Sandbox
Files may be copied from/to local filesystems (WNs, UIs) to the Grid (SEs)

5. OVERVIEW
Storage Element is the service which saves/loads files to/from local storages. These local storages can be both, a disk or large storage systems.
Functions:
File storage.
Storage resources administration interface.
Storage space administration.
gLite 3.1 data access protocols:
File Transfer: GSIFTP (GridFTP)
File I/O (Remote File access): gsidcap
insecure RFIO
secured RFIO (gsirfio)

6. OVERVIEW Classic SE: GridFTP server
Insecure RFIO daemon (rfiod) – only LAN limited file access
Single disk or disk array
No quota management
Will not be supported anymore
Mass Storage Systems (Castor)
Files migrated between front-end disk and back-end tape storage hierarchies
Insecure RFIO (Castor)
Provide a SRM interface with all the benefits
Disk pool managers (dCache and gLite DPM)
manage distributed storage servers in a centralized way
Physical disks or arrays are combined into a common (virtual) file system
Disks can be dynamically added to the pool
Secure remote access protocols (gsidcap for dCache, gsirfio for DPM)
SRM interface

7. OVERVIEW
The Disk Pool Manager (DPM) is a lightweight solution for disk storage management, which offers the SRM (Storage Resource Manager) interfaces.
Each DPM–type Storage Element (SE), is composed by a head node and a disk server on the same machine.
The DPM head node has to have at least one files system in the pool, and then, an arbitrary number of disk servers can be added by YAIM.
The DPM handles the storage on Disk Servers. It handles pools: a pool is a group of file systems, located in one or more disk servers. The DPM Disk Servers can have multiple files system in the pool.
arbitrary 随意的，专横的

8. OVERVIEW Usually the DPM head node hosts:
SRM server (srmv1 and/or srmv2) : receives the SRM requests and passes them to the DPM server.
DPM server : keeps track of all the requests.
DPM name server (DPNS) : handles the namespace for all the files under the DPM control.
DPM RFIO server : handles the transfers for the RFIO protocol.
DPM Gridftp server : handles the transfer for the Gridftp protocol.

9. OVERVIEW File Catalog Motivations
Users and programs produce and require data
Data may be stored in Grid datasets (files)
Located in Storage Elements (SEs)
Several replicas of one file in different sites
Accessible by Grid users and applications from “anywhere”
Locatable by the WMS (data requirements in JDL)
Data may be copied from/to local filesystems (WNs, UIs) to the Grid
Why File Catalogs are used?
They keep track of the location of copies (replicas) of Grid files
They provide DM tools and APIs through which users and grid services can deal with files

10. OVERVIEW Logical File Name (LFN) Globally Unique Identifier (GUID)
An alias created by a user to refer to some item of data, e.g. “lfn:/grid/gilda/tutorials/zhuw/text_file.txt”
Globally Unique Identifier (GUID)
A non-human-readable unique identifier for an item of data, e.g.
“guid:f81d4fae-7dec-11d0-a765-00a0c91e6bf6”
Site URL (SURL) (or Physical File Name (PFN) or Site FN)
The location of an actual piece of data on a storage system, e.g. “srm://gilda03.ihep.ac.cn/dpm/ihep.ac.cn/home/gilda/generated/ / filef7c237f9-15ff-4cd1-80d8-f2ae33199b51” (lcg-lr they can be many if the file has replicas somewhere)
Transport URL (TURL)
Temporary locator of a replica + access protocol: understood by a SE, e.g.
“rfio://gilda03.ihep.ac.cn//storage/gilda/ /filef7c237f9-15ff-4cd1-80d8-f2ae33199b ”

11. INSTALLATION & CONFIGURATION

12. Preliminaries
The instructions provided in this presentation assume that you are going to use a machine with Scientific Linux 5.4 installed.
we are using the Scientific Linux 5.4 x86_64.

13. REPOS=“dag ig glite-generic lcg-ca glite-se_dpm_mysql”
Repository settings
Specify the mrepo host:
export MREPO=
Configure the repository as follows:
REPOS=“dag ig glite-generic lcg-ca glite-se_dpm_mysql”
Get repositories with:
for name in $REPOS;
do wget $MREPO/$name.repo -O /etc/yum.repos.d/$name.repo; done
Copy the gilda utils:
wget -O /etc/yum.repos.d/gilda.repo
To get a full mirror of GILDA repository:
wget -mp -nH -np
Follows a brief explanations of the used parameters:
-m This option is used to make mirrors. This means that the next time the command will be issued it will download just the updated files rather than the whole site again.
-p It will include other elements like images to correctly show the mirrored web pages
-nH This command avoids to create the subdirectories: ‘grid018.ct.infn.it/rep’ onthe destination
Follows a brief explanations of the used parameters:-mThis option is used to make mirrors. This means that the next time the command will be issued it will download just the updated files rather than the whole site again.-pIt will include other elements like images to correctly show the mirrored web pages -nHThis command avoids to create the subdirectories: ‘grid018.ct.infn.it/rep’ onthe destination
Then replace baseurl filed in any .repo file inside repos/ directorty
#!/bin/bash #
# Change repo host
MREPO_FROM=
MREPO_TO=
MREPO_FROM_ESCAPE=$(echo $MREPO_FROM | sed s/'\/'/'\\\/'/g)
MREPO_TO_ESCAPE=$(echo $MREPO_TO | sed s/'\/'/'\\\/'/g)
for repo in $(ls -1 *.repo) do
echo $repo
mv $repo $repo.todo
cat $repo.todo | sed s/"${MREPO_FROM_ESCAPE}"/"${MREPO_TO_ESCAPE}"/g > $repo
rm -f $repo.todo
done

14. System requirements Please check: If you have a valid hostname
To verify, type: hostname -f
To edit the hostname:
vim /etc/hosts
vim /etc/sysconfig/network
hostname <pc_name>

15. yum install ig_SE_dpm_mysql yum install gilda_utils
Then update your host:
yum clean all
Install CAs on ALL profiles:
yum install lcg-CA
Metapackage installation:
yum install ig_SE_dpm_mysql
Gilda utils:
yum install gilda_utils

16. site-info.def Copy the ig-site-info.def script template
Open your own site-info.def and customize it:
cp /opt/glite/yaim/examples/siteinfo/ig-site-info.def /opt/glite/yaim/examples/siteinfo/mysite-info-se-dpm.def
vi /opt/glite/yaim/examples/siteinfo/mysite-info-se-dpm.def

17. site-info.def
USERS_CONF=${INSTALL_ROOT}/glite/yaim/examples/ig-users.conf
GROUPS_CONF=${INSTALL_ROOT}/glite/yaim/examples/ig-groups.conf
MYSQL_PASSWORD=123456
JAVA_LOCATION="/usr/lib/jvm/java openjdk x86_64/jre“
SITE_NAME=my-site-name (haven’t defined it yet)
SITE_ =“your address“
PX_HOST=myproxy.ct.infn.it
DPM_HOST=“your DPM hostname“
SE_LIST=“your DPM hostname“
BDII_HOST=topBDII hostname
Add # before SITE_BDII_HOST=my-bdii.$MY_DOMAIN

18. site-info.def Maybe you need add these line below：
#########################################
# StoRM backend configuration variables #
# More information here:
# StoRM backend host
STORM_BACKEND_HOST=my-storm-backend.$MY_DOMAIN
# StoRM storage path
STORM_DEFAULT_ROOT=/storage

19. CONFIGURATION
In the machine of gilda**.ihep.ac.cn , the host certifcates locate at :/root/gilda12- 28certs.tar
#tar -xvf gilda12-28certs.tar
Then your host certificates are in the directory named your hostname
Install host certificate:
Download your certificates in /etc/grid-security:
# cd gilda12-28certs
#mv gilda**/hostcert.pem /etc/grid-security/hostcert.pem
#mv gilda**/hostkey.pem /etc/grid-security/hostkey.pem
and set proper permissions:
#chmod 644 /etc/grid-security/hostcert.pem
#chmod 400 /etc/grid-security/hostkey.pem

20. CONFIGURATION #cd /opt/glite/yaim/examples #vi ig-groups.conf
Add "/gilda/ROLE=SoftwareManager":: gm:
"/gilda/grelc/das/*":gilda:::
"/gilda"::::
~]# vi /etc/hosts.allow
Add one line:
slapd: all
#cd /opt/glite/yaim/examples/siteinfo/services
#vi glite-se_dpm_mysql
DPMPOOL="Permanent“
DPM_FILESYSTEMS="$DPM_HOST:/storage“
DPM_DB_USER=dpmdbuser
DPM_DB_PASSWORD=123456
DPM_DB_HOST=$DPM_HOST
DPM_INFO_USER="dpminfo”
DPM_INFO_PASS=secret

21. /etc/init.d/ntpd status /etc/init.d/ntpd restart
Configuring the SE for GILDA
Update the date
/etc/init.d/ntpd status
/etc/init.d/ntpd restart
Configure the UI with YAIM
/opt/glite/yaim/bin/ig_yaim -c -s /opt/glite/yaim/examples/siteinfo/mysite-info-se-dpm.def -n ig_SE_dpm_mysql
If everithing is ok, the SE is now READY for testing

22. TESTING

23. TESTING Login UI with your account [zhuw@gilda08 ~]$ dpm-qryconf
POOL Permanent DEFSIZE M GC_START_THRESH 0 GC_STOP_THRESH 0 DEF_LIFETIME 7.0d DEFPINTIME 2.0h MAX_LIFETIME 1.0m MAXPINTIME 12.0h FSS_POLICY maxfreespace GC_POLICY lru RS_POLICY fifo GIDS 0 S_TYPE - MIG_POLICY none RET_POLICY R
CAPACITY 32.99G FREE 26.98G ( 81.8%)
gilda03.ihep.ac.cn /storage CAPACITY 32.99G FREE 26.98G ( 81.8%)
~]$ dpns-ls /
dpm
~]$ dpns-ls /dpm/
ihep.ac.cn
~]$ dpns-ls /dpm/ihep.ac.cn/
home
~]$ dpns-ls /dpm/ihep.ac.cn/home
gilda
…………

24. TESTING rfio [zhuw@gilda08 ~]$ rfdir /dpm/ihep.ac.cn/home/gilda
drwxrwxr-x Aug 29 16:07 generated
-rw-rw-r Aug 23 00:28 malx-1.jdl
-rw-rw-r Aug 25 23:28 malx-21.jdl
-rw-rw-r Aug 20 18:34 malx.jdl
-rw-rw-r Sep 01 00:46 tutorials
~]$
~]$ rfcp /dpm/ihep.ac.cn/home/gilda/malx-1.jdl /tmp/zhuw.jdl
115 bytes in 0 seconds through eth0 (in) and local (out)
~]$ ll /tmp/zhuw.jdl
-rw-rw-r-- 1 zhuw zhuw 115 Sep 1 01:04 /tmp/zhuw.jdl

25. TESTING [zhuw@gilda08 ~]$ lcg-infosites --vo gilda se
Avail Space(Kb) Used Space(Kb) Type SEs
n.a vega-se.ct.infn.it
n.a dgt02.ui.savba.sk
…………
n.a aliserv6.ct.infn.it
n.a gilda03.ihep.ac.cn
n.a se.hpc.iit.bme.hu
n.a se1-egee.srce.hr
~]$
~]$ ldapsearch -x -h gilda03.ihep.ac.cn -p b mds-vo-name=resource,o=grid | grep AvailableSpace
GlueSAStateAvailableSpace: