1. Installazione, configurazione e testing di Storage Element SE [DPM] + MON
Federico Bitelli
bitelli<at>fis.uniroma3.it
Dipartimento di Fisica Università Roma Tre e
INFN-ROMA3
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste Novembre 2008
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

2. Outline Disk Pool Manager Overview How to install a DPM Server
DPM testing
Part 2
Introduction to R-GMA
How to install a Mon Box
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

3. Disk Pool Manager Overview
The Disk Pool Manager (DPM) is a lightweight solution for disk storage management, which offers the SRM interfaces.
It may act as a replacement for the obsolete classical SE with the following advantages :
 SRM interface (both v1.1 and v2.2)
 Better scalability : DPM is allow to manage 100+ TB distributing the load over several servers
High performances
 Light-weight management
The DPM head node has to have one filesystem in this pool, and then an arbitrary number of disk servers can be added by YAIM.
The DPM disk servers can have multiple filesystems in the pool.
The DPM head node also hosts the DPM and DPNS databases, as well as the SRM web service interfaces.
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

4. II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

5. DPM architecture /dpm /domain /home CLI, C API, /vo
SRM-enabled client, etc.
/vo
DPM
head node
file
DPM Name Server
Namespace
Authorization
Physical files location
DPM Server
Requests queuing and processing
Space management
SRM Servers (v1.1, v2.1, v2.2)
Disk Servers
Physical files
Direct data transfer from/to
disk server (no bottleneck)
data transfer …
DPM
disk servers
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

6. Usually the DPM head node hosts:
SRM server (srmv1 and/or srmv2) : receives the SRM requests and pass them to the DPM server;
DPM server : keeps track of all the requests;
DPM name server (DPNS) : handles the namespace for all the files under the DPM control;
DPM RFIO server : handles the transfers for the RFIO protocol;
DPM Gridftp server : handles the transfer for the Gridftp protocol.
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

7. Installing DPM
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

8. Installing pre-requisites /1
Start from a fresh install of SLC 4.X
Installation will install all dependencies, including
other necessary gLite modules
external dependencies
Repository for all metapackages
dag.repo
jpackage.repo
ig.repo
lcg-ca.repo
Repository for DPM disk
glite-se_dpm_disk.repo
Repository for DPM_Mysql
glite-se_dpm.repo
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

9. 1. Download the *.repo files from repository:
>export REPOS="ca dag glite-ui ig jpackage glite- se_dpm_disk glite-se_dpm.repo gilda”
>for name in $REPOS; do wget -O /etc/yum.repos.d/$name.repo; done
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

10. Installing pre-requisites /2
We need a dedicated partition for the storage area
Check the partition
# df –h
Filesystem Size Used Avail Use% Mounted on
/dev/sda G 820M 8.4G 9% /
/dev/sda G 33M 18G 1% /storage
none M M 0% /dev/shm
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

11. Installing pre-requisites /3
Syncronization among all gLite nodes is mandatory.
So install ntp
#yum install ntp
You can check ntpd’s status with:
#ntpq –p
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

12. Installing pre-requisites /4
Check the FQDN (fully qualified domain name) hostname
Ensure that the hostnames of your machines are correctly set. Run the command:
#hostname –f
if your hostname is incorrect :
edit the file /etc/sysconfig/network and set the HOSTNAME variable.
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

13. Installing pre-requisites /5
Request host certificates for the SE to your RA
Copy host certificate (hostcert.pem and hostkey.pem) in /etc/grid-security.
Change files permission
#chmod 644 /etc/grid-security/hostcert.pem
#chmod 400 /etc/grid-security/hostkey.pem
If you plan to use certificates released by unsupported EGEE CA’s, be sure that their public key (usually distributed with a rpm) are installed.
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

14. Installation #yum clean all #yum update
#yum install java sun-compat
Install the CAs
#yum install lcg-CA
(For this tutorial add the pakage gilda_utils)
#yum install gilda_utils
Install the metapackage
– yum install <metapackage>:
#yum install ig_SE_dpm_disk
#yum install ig_SE_dpm_mysql
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

15. DPM Configuration #ls /sitedir/services
Create a copy of /opt/glite/yaim/examples/site- info.def template to your reference directory for the installation (e.g. /root/sitedir):
cp /opt/glite/yaim/examples/site-info.def /root/sitedeir/site- info.def
Copy the directory ‘services’ in the same location
# ls /root/sitedir/
my-ig-site-info.def services
#ls /sitedir/services
glite-se_dpm_disk glite-se_dpm_mysql glite-se_dpm_oracle
ig-hlr
Edit the site-info.def file
A good syntax test for your site configuration file is to try to source it manually running the command:
#source site-info.def
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

16. site.def MY_DOMAIN = grid.vm your domain name (check it)
MON_HOST = mon.grid.vm FQDN of MON Box
JAVA_LOCATION="/usr/java/j2sdk1.4.2_16“ #java location Check your version
MYSQL_PASSWORD=passwd_root #the root Mysql password
VOS="gilda infngrid" The VO we want …
ALL_VOMS_VOS=“gilda infngrid “
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

17. site.def (DPM) DPM Variables in site-info.def
DPM_HOST= <your host>.$MY_DOMAIN #FQDN of DPM head node
DPM_DB_USER=dpmmgr #The user for our database
DPM_DB_PASSWORD=my_sql_pass MYSQL password
DPMFSIZE=200M # The space to be reserved by default for a file stored in the DPM
DPMPOOL=Permanent #**The name and type of the pool including file system(ex: Permanent)
DPM_FILESYSTEMS="$DPM_HOST:/storage" # The filesystems parts of the pool ‘
SE_GRIDFTP_LOGFILE=/var/log/dpm-gsiftp/dpm-gsiftp.log (Check it ?
**The DPM can handle two #different kinds of file systems:
* volatile : the files contained in a volatile file system can be removed by the system at any time,
unless they are pinned by a user.
* permanent : the files contained in a permanent file system cannot be removed by the system.
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

18. Firewall configuration
The following ports have to be open:
DPM server: port 5015/tcp must be open locally at your site at least (can be incoming access as well),
DPNS server: port 5010/tcp must be open locally at your site at least (can be incoming access as well),
SRM servers: ports 8443/tcp (SRMv1) and 8444/tcp (SRMv2) must be opened to the outside world (incoming access),
RFIO server: port 5001/tcp must be open to the outside world (incoming access), in the case your site wants to allow direct RFIO access from outside,
Gridftp server: control port 2811/tcp and data ports /tcp (or any range specified by GLOBUS_TCP_PORT_RANGE) must be opened to the outside world (incoming access).
FOR THIS TUTORIAL JUST STOP IPTABLES
#service iptables stop
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

19. Middleware Configuration
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> - n ig_SE_dpm_mysql
/opt/glite/yaim/bin/ig_yaim -c -s <your-site-info.def> - n ig_SE_dpm_disk
If you want install the disks on another machine you can run
/opt/glite/bin/ig_install_node /root/site-info.def ig_SE_dpm_disk
/opt/glite/bin/ig_configure_node site-info.def ig_SE_dpm_disk
on the other machine
Then run on (dpm_mysql machine)
dpm-addfs --poolname Permanent --server diskserverhostname –fs /storage2
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

20. After configuration remember to manually run the script /etc/cron
After configuration remember to manually run the script /etc/cron.monthly/create-default-dirs-DPM.sh as suggested by yaim log. This script create and set the correct permissions on VO storage directories; it will be run monthly via cron.
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

21. DPM Server Testing
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

22. Testing DPM
A simple test for checking if the DPM server is correctly exporting the filesystem is:
dpm-qryconf
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

23. Post configuration Login into the UI: Set the variables:
DPM_HOST : “export DPM_HOST=<your DPM host>”
DPNS_HOST : “export DPNS_HOST=<your DPM host>”
Execute following commands :
dpm-qryconf
dpns-ls /
dpns-mkdir
dpns-rm
#uberftp se1.grid.box (chek if this works!)
Then try to really copy a file using globus
# globus-url-copy file:/tmp/myfile gsiftp://se1.grid.box/dpm/grid.box/home/gilda/testfile
If you have a FTS Server u could try to use your new DPM as endpoint using
glite-transfer-submit
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

24. Other command to build NameSpace dpns-mkdir dpns-chmod dpns-chown
dpns-setacl
And commands to add pools and filesystems
dpm-addfs
dpm-addpool
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

25. Log-files If you have some problem try to analyze your log-files
/var/log/dpns/log
/var/log/dpm/log
/var/log/dpm-gsiftp/dpm-gsiftp.log
/var/log/srmv1/log
/var/log/srmv2/log
/var/log/srmv2.2/log
/var/log/rfio/log
SE_GRIDFTP_LOGFILE=/var/log/dpm-gsiftp/dpm-gsiftp.log
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

26. And take a look at mysql db #mysql –p –u dpmmgr Enter password:*****
| Tables_in_dpm_db |
| dpm_copy_filereq |
| dpm_fs |
| dpm_get_filereq |
| dpm_pending_req |
| dpm_pool |
| dpm_put_filereq |
| dpm_req |
| dpm_space_reserv |
| dpm_unique_id |
| schema_version |
And take a look at mysql db
#mysql –p –u dpmmgr
Enter password:*****
mysql> show databases;
| Database |
| cns_db |
| dpm_db |
| mysql |
| test |
| Tables_in_cns_db |
| Cns_class_metadata |
| Cns_file_metadata |
| Cns_file_replica |
| Cns_groupinfo |
| Cns_symlinks |
| Cns_unique_gid |
| Cns_unique_id |
| Cns_unique_uid |
| Cns_user_metadata |
| Cns_userinfo |
| schema_version |
mysql>connect cns_db;
mysql>show tables;
mysql>connect dpm_db;
mysql>show tables;
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

27. Reference http://www.gridpp.ac.uk/wiki/Disk_Pool_Manager
II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November

28. II Corso di formazione INFN per amministratori di siti Grid
ICTP, Trieste November