1. www.epikh.eu The EPIKH Project (Exchange Programme to advance e-Infrastructure Know-How) WMS LB BDII Installation and Configuration Salma Saber (salma_saber@eri.sci.eg) ERI-Cairo, Egypt Joint EPIKH/EUMEDGRID Support event in Cairo Africa 4 – Site Admin Tutorial + Remote site inst. Egypt, 17.10.2010

2. o The Workload Management System (WMS) is the gLite service that allows users to submit jobs, and performs all tasks required to execute them, without exposing the user to the complexity of the Grid. o The workload manager (WM) is the principal component in the WMS. o Purpose of the Workload Manager (WM) is to accept and satisfy requests for job management coming from its clients o The WM hands over the job to an appropriate Computing Element (CE) for execution taking into account requirements and the preferences expressed in the job description. 2 Workload Management System (WMS) Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

3. o The Logging and Bookkeeping service (LB) tracks jobs in terms of event (important points of job life, e.g. submission, finding a matching CE, starting execution etc.) gathered from various WMS components as well as CEs. 3 Logging and Bookkeeping service (LB) Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

4. 4 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

5. 5 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Acts as the portal for accepting all external requests to the workload manager. Job submission, cancellation expressed via a JDL

6. 6 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Keeps submission requests Requests are kept for a while for a while if no resources are immediately available

7. 7 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Finds an appropriate CE for each submission request, taking into account job requests and preferences, Grid status, utilization policies on resources

8. 8 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 A cache where overall resource information about the grid is stored for querying by the matchmaker. Updating these info via the information updater.

9. 9 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Updating of information is carried out by reading information from the top- level BDII server of the virtual organization.

10. 10 gLite WMS Architecture Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 The LB is responsible to: - Stores events generated by the various components of the WMS - Querying the LB user can retrieve information about the status of the job

11. o The instructions provided in this presentation assume that you are going to use a machine with Scientific Linux 4.8 installed.  Virtual Machine image (GILDA VM Base_SLC4.8_i386) available at: https://gilda.ct.infn.it/VirtualServices.html https://gilda.ct.infn.it/VirtualServices.html 11 Preliminaries Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

12. To login to the VM you can use the winscp then open a terminal and write the password Password: cairoschool 12 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

13. o Specify the mrepo host: export MREPO=http://server1.eun.eg/mrepo/repo/ o Configure the repository as follows: REPOS="dag ig glite-generic lcg-ca jpackage glite-bdii glite-wms" o Get repositories with: for name in $REPOS; do wget $MREPO/$name.repo -O /etc/yum.repos.d/$name.repo; done o Copy the gilda utils (Note: only for Gilda VO): wget http://grid018.ct.infn.it/mrepo/repos/gilda.repo -O /etc/yum.repos.d/gilda.repo 13 Repository settings Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

14. o Let’s check if date’s machine is correct with: date o if ntp date isn’t correct: /etc/init.d/ntpd status ntpdate ntp-1.infn.it o if not let’s configure file and make service start on boot: /etc/init.d/ntpd start chkconfig ntpd on 14 Network Time Protocol Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

15. o Please check: –If you have a valid hostname  To verify, type: hostname -f  To edit the hostname: vi /etc/hosts vi /etc/hostname vi /etc/sysconfig/network hostname 15 System requirements Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

16. o Modify the ig.repo as following: (vi /etc/yum.repos.d/ig.repo) # # INFNGRID repositories # [ig_sl4_i386] name = INFNGRID 3.1 i386 baseurl = http://server1.eun.eg/mrepo/ig_sl4-i386/RPMS.3_1_0/ enabled = 1 protect = 0 [ig_sl4_i386_externals] name = INFNGRID 3.1 i386 (externals) baseurl = http://server1.eun.eg/mrepo/ig_sl4-i386/RPMS.3_1_0_externals/ enabled = 0 protect = 0 16 Some Modifications Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

17. o Modify the glite-generic.repo as following: (vi /etc/yum.repos.d/glite-generic.repo) # # gLite generic repositories # [glite-generic_sl4_i386_release] name = gLite generic 3.1 i386 (release) baseurl = http://server1.eun.eg/mrepo/glite_ig_sl4-i386/RPMS.generic-release/ enabled = 1 protect = 0 [glite-generic_sl4_i386_updates] name = gLite generic 3.1 i386 (updates) baseurl = http://server1.eun.eg/mrepo/glite_ig_sl4-i386/RPMS.generic-updates/ enabled = 0 protect = 0 17 Some Modifications Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

18. o Then update your host: yum clean all o Install CAs on ALL profiles: yum install lcg-CA o Ig: yum install ig-yaim o Gilda utils: yum install gilda_utils 18 YUM install Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

19. o WMS install: yum install ig_WMS 19 WMS installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

20. Transaction Check Error 20 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Transaction Check Error: file /opt/glite/lib/libvomsapi_nog.a conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security-voms-api-noglobus-1.8.8-2.slc4 file /opt/glite/lib/libvomsapi_nog.so conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security-voms-api- noglobus-1.8.8-2.slc4 file /opt/glite/lib/libvomsapi_nog.so.0 conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security-voms-api- noglobus-1.8.8-2.slc4 file /opt/glite/lib/libvomsapi_nog.so.0.0.0 conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security- voms-api-noglobus-1.8.8-2.slc4 Transaction Check Error: file /opt/glite/lib/libvomsapi_nog.a conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security-voms-api-noglobus-1.8.8-2.slc4 file /opt/glite/lib/libvomsapi_nog.so conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security-voms-api- noglobus-1.8.8-2.slc4 file /opt/glite/lib/libvomsapi_nog.so.0 conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security-voms-api- noglobus-1.8.8-2.slc4 file /opt/glite/lib/libvomsapi_nog.so.0.0.0 conflicts between attempted installs of glite-security-voms-api-cpp-1.9.10-8.slc4 and glite-security- voms-api-noglobus-1.8.8-2.slc4 If continuing having this error try yum remove glite-security-voms-cpp

21. o This error will appear during the installation of ig_WMS: rpm -ivh http://google-perftools.googlecode.com/files/google-perftools-1.5-1.i386.rpm rpm -ivh http://google-perftools.googlecode.com/files/google-perftools-devel-1.5- 1.i386.rpm yum install ig_WMS 21 Yum Install Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Error: Missing Dependency: google-perftools >= 1.0-1 is needed by package ig_WMS Error: Missing Dependency: google-perftools-devel >= 1.0-1 is needed by package ig_WMS Error: Missing Dependency: google-perftools >= 1.0-1 is needed by package ig_WMS Error: Missing Dependency: google-perftools-devel >= 1.0-1 is needed by package ig_WMS

22. cd /etc/yum.repos.d/ wget http://server1.eun.eg/mrepo/repo/glite-lb.repohttp://server1.eun.eg/mrepo/repo/glite-lb.repo yum clean all yum install ig_LB 22 LB installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

23. o BDII install: yum install ig_BDII 23 Top BDII installation Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

24. o Move the host cert to the right path mv /root/SRVXX.eun.eg/SRVXX.eun.eg-cert.pem /etc/grid-security/hostcert.pem mv /root/SRVXX.eun.eg/SRVXX.eun.eg-key.pem /etc/grid-security/hostkey.pem o Give them the right permissions chmod 400 /etc/grid-security/hostkey.pem chmod 644 /etc/grid-security/hostcert.pem [root@cerist04 ~]# ll /etc/grid-security/ total 32 drwxr-xr-x 2 root root 20480 Jun 27 12:08 certificates -rw-r--r-- 1 root root 887 Jun 16 10:02 hostcert.pem -r-------- 1 root root 887 Jun 16 10:02 hostkey.pem drwxr-xr-x 2 root root 4096 Jun 27 11:21 vomsdir 24 Host cetificates Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

25. o generate configurations file from YAIM template cp /opt/glite/yaim/examples/siteinfo/ig-site- info.def /opt/glite/yaim/examples/siteinfo/mysite- info.def o Open your own site-info.def and customize it: vi /opt/glite/yaim/examples/siteinfo/mysite-info.def 25 site-info.def Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

26. 26 WMS-LB Configuration (1/2) WMS Configuration INSTALL_ROOT=/opt MYSQL_PASSWORD=secret PX_HOST=myproxy.ct.infn.it WMS_HOST=SRVXX.eun.eg SITE_EMAIL= VOS="gilda"  Add gilda here ######### # gilda # ######### VO_GILDA_VOMS_CA_DN="/C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it" VO_GILDA_SW_DIR=$VO_SW_DIR/gilda VO_GILDA_DEFAULT_SE=$DPM_HOST VO_GILDA_STORAGE_DIR=$CLASSIC_STORAGE_DIR/gilda VO_GILDA_VOMS_SERVERS="voms://voms.ct.infn.it:8443/voms/gilda?/gilda" VO_GILDA_VOMSES="gilda voms.ct.infn.it 15001 /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it gilda" Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

27. o Open glite-wms file and edit: vi /opt/glite/yaim/examples/siteinfo/services/glite-wms LB_HOST= “SRVXX.eun.eg:9000" GLITE_WMS_LCGMON_FILE="/var/log/wms_lcgmon.log" o Copy the gilda profiles inside ig ones cat /opt/glite/yaim/etc/gilda/gilda_ig-groups.conf >> /opt/glite/yaim/examples/ig-groups.conf cat /opt/glite/yaim/etc/gilda/gilda_ig-users.conf >> /opt/glite/yaim/examples/ig-users.conf o CONFIGURE /opt/glite/yaim/bin/ig_yaim -c -s /opt/glite/yaim/examples/siteinfo/mysite-info.def -n ig_WMS -n ig_LB 27 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 WMS-LB Configuration (2/2)

28. 28 /opt/glite/etc/init.d/glite-wms-wmproxy: line 37: 6431 Segmentation fault $httpdpath -k $@ -f $config ABORT: Service glite-wms-wmproxy failed to start! ERROR: Error during the execution of function: config_glite_wms ERROR: Error during the configuration.Exiting. [FAILED] /opt/glite/etc/init.d/glite-wms-wmproxy: line 37: 6431 Segmentation fault $httpdpath -k $@ -f $config ABORT: Service glite-wms-wmproxy failed to start! ERROR: Error during the execution of function: config_glite_wms ERROR: Error during the configuration.Exiting. [FAILED] # rpm -qa |grep gridsite gridsite-apache-1.1.18.1-1 gridsite-shared-1.1.18.1-1 # rpm -e --nodeps gridsite-apache-1.1.18.1-1 gridsite-shared-1.1.18.1-1 # wget http://etics- repository.cern.ch/repository/download/registered/org.glite/org.gridsite.core/1.1.20/slc4_ia32_ gcc346/gridsite-apache-1.1.20-2.i386.rpm # wget http://etics- repository.cern.ch/repository/download/registered/org.glite/org.gridsite.core/1.1.20/slc4_ia32_ gcc346/gridsite-shared-1.1.20-2.i386.rpm # rpm -ivh gridsite-shared-1.1.20-2.i386.rpm # rpm -ivh gridsite-apache-1.1.20-2.i386.rpm THEN CONFIGURE AGAIN Configuration Error Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

29. 29 Testing WMS-LB From a UI enter: [elingra@glite-tutor ~]$ voms-proxy-init –voms gilda [elingra@glite-tutor ~]$ export GLITE_WMS_WMPROXY_ENDPOINT=https://SRVXX.eun.eg:7443/glite_w ms_wmproxy_server o Create a.jdl file: [elingra@glite-tutor ~]$ vi hostname.jdl Type = "Job"; JobType = "Normal"; Executable = "/bin/hostname"; StdOutput = "hostname.out"; StdError = "hostname.err"; OutputSandbox = {"hostname.err","hostname.out"}; Arguments = "-f"; ShallowRetryCount = 3; Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

30. 30 Testing WMS-LB [elingra@glite-tutor ~]$ glite-wms-job-list-match -a hostname.jdl Connecting to the service https://grid-test-60.trigrid.it:7443/glite_wms_wmproxy_server ========================================================================== COMPUTING ELEMENT IDs LIST The following CE(s) matching your job requirements have been found: *CEId* - grisuce.scope.unina.it:2119/jobmanager-lcgpbs-grisu_long - grisuce.scope.unina.it:2119/jobmanager-lcgpbs-grisu_short - ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_long - ce1-egee.srce.hr:2119/jobmanager-sge-prod - gilda-01.pd.infn.it:2119/jobmanager-lcgpbs-gilda - ce-edu.grid.acad.bg:2119/jobmanager-pbs-gilda - ce.scope.unina.it:2119/jobmanager-lcgpbs-egee_short - sirius-ce.ct.infn.it:2119/jobmanager-lcgpbs-gilda - grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda - iceage-ce-01.ct.infn.it:2119/jobmanager-lcgpbs-gilda ========================================================================== Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

31. [root@gilda11 examples]# service gLite status *** globus-gridftp: globus-gridftp-server (pid 13095) is running... *** glite-wms-wmproxy: WMProxy httpd listening on port 7443 httpd (pid 26197 12937 12936 12935 12934 12933 12932) is running.... === WMProxy Server running instances: UID PID PPID C STIME TTY TIME CMD *** glite-wms-wm: /opt/glite/bin/glite-wms-workload_manager (pid 12466) is running... *** glite-wms-lm: Logmonitor running... *** glite-wms-jc: JobController running in pid: 12525 CondorG master running in pid: 12581 CondorG schedd running in pid: 12662 *** glite-wms-ice: /opt/glite/bin/glite-wms-ice-safe (pid 12636) is running... *** glite-proxy-renewald: glite-proxy-renewd running *** glite-lb-proxy: glite-lb-proxy running as 13250 *** glite-lb-locallogger: glite-lb-logd running glite-lb-interlogd not running *** glite-lb-bkserverd: glite-lb-notif-interlogd running glite-lb-bkserverd running as 13764 31 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010 Testing WMS-LB

32. 32 BDII Configuration BDII_HOST=SRVXX.eun.eg SITE_NAME=GILDA_EGYPT BDII_HTTP_URL= " http://SRVXX.eun.eg/fileadmin/bdii/gl ite-bdii-update.conf " NTP_HOSTS_IP="193.206.144.10 ntp-1.infn.it" VOS= "gilda" Then configure: /opt/glite/yaim/bin/ig_yaim -c -s /opt/glite/yaim/examples/siteinfo/mysite-info.def -n ig_BDII_top BDII Configuration Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

33. 33 BDII Post Configuration Set the CA autoupdate on /opt/glite/yaim/etc/gilda/set_CAs_autoupdate.sh vi /opt/bdii/etc/bdii.conf Then Edit: BDII_AUTO_UPDATE=yes BDII_MODIFY_DN=yes BDII_UPDATE_URL=http://SRVXX.eun.eg/fileadmin/bdii/g lite-bdii-update.conf Restart BDII /etc/init.d/bdii restart Test bdii ldapsearch -x -h localhost -p 2170 -b mds-vo- name=local,o=grid Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

34. In order to customize the Site BDII list (http://SRVXX.eun.eg/fileadmin/bdii/glite-bdii-update.conf) install the htppd service yum install -y httpd (if necessary) Publish your customized glite-bdii-update.conf file (i.e. /var/www/html/bdii) Set the BDII_UPDATE_URL accordingly in the file opt/bdii/etc/bdii.conf file Restart the BDII service /etc/init.d/bdii restart 34 Customize your BDII Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010

35. 35 References  INFNGrid installation http://igrelease.forge.cnaf.infn.it/doku.php?id=doc:guides:install-3_1 Cairo, Joint EPIKH/EUMEDGRID Support Site Admin Tutorial 17.10.2010