Presentation is loading. Please wait.

Presentation is loading. Please wait.

Www.eu-eela.org E-science grid facility for Europe and Latin America gLite MPI Tutorial for Grid School Daniel Alberto Burbano Sefair, Universidad de Los.

Similar presentations


Presentation on theme: "Www.eu-eela.org E-science grid facility for Europe and Latin America gLite MPI Tutorial for Grid School Daniel Alberto Burbano Sefair, Universidad de Los."— Presentation transcript:

1 www.eu-eela.org E-science grid facility for Europe and Latin America gLite MPI Tutorial for Grid School Daniel Alberto Burbano Sefair, Universidad de Los Andes (Bogotá, Colombia) UNAM, Querétaro, México 28 September to 10 October 2009

2 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 gLite MPI (Outline) Requierements to submit jobs. Get information from the Grid. Structure of a MPI job in the Grid with mpi-start. Structure of MPI jobs without mpi-start. 2

3 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Requierements to submit jobs MPI-1 is for MPICH and LAM. MPI-2 is for OpenMPI and MPICH-2. Proxy certificate. Permissions to submit job in a specific site. The mpi jobs can run only in one Grid site. 3

4 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Get information from the Grid Find the sites that support MPICH and MPICH2 4 [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPICH‘ - CE: ce-eela.ciemat.es:2119/jobmanager-lcgpbs-prod_eela - CE: ce01.eela.if.ufrj.br:2119/jobmanager-lcgpbs-prod - CE: grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday - CE: kuragua.uniandes.edu.co:2119/jobmanager-lcgpbs-prod [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPICH‘ - CE: ce-eela.ciemat.es:2119/jobmanager-lcgpbs-prod_eela - CE: ce01.eela.if.ufrj.br:2119/jobmanager-lcgpbs-prod - CE: grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday - CE: kuragua.uniandes.edu.co:2119/jobmanager-lcgpbs-prod [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPICH2‘ - CE: grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPICH2‘ - CE: grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday

5 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Get information from the Grid Find the sites that support MPICH and its available CPUs 5 [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPICH' --attrs 'CE,FreeCPUs,TotalCPUs' - CE: ce-eela.ciemat.es:2119/jobmanager-lcgpbs-prod_eela - CE ce-eela.ciemat.es:2119/jobmanager-lcgpbs-prod_eela - FreeCPUs 155 - TotalCPUs 220 - CE: ce01.eela.if.ufrj.br:2119/jobmanager-lcgpbs-prod - CE ce01.eela.if.ufrj.br:2119/jobmanager-lcgpbs-prod - FreeCPUs 239 - TotalCPUs 240 - CE: grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - CE grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - FreeCPUs 22 - TotalCPUs 24 - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - FreeCPUs 522 - TotalCPUs 748 - CE: kuragua.uniandes.edu.co:2119/jobmanager-lcgpbs-prod - CE kuragua.uniandes.edu.co:2119/jobmanager-lcgpbs-prod - FreeCPUs 71 - TotalCPUs 84 [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPICH' --attrs 'CE,FreeCPUs,TotalCPUs' - CE: ce-eela.ciemat.es:2119/jobmanager-lcgpbs-prod_eela - CE ce-eela.ciemat.es:2119/jobmanager-lcgpbs-prod_eela - FreeCPUs 155 - TotalCPUs 220 - CE: ce01.eela.if.ufrj.br:2119/jobmanager-lcgpbs-prod - CE ce01.eela.if.ufrj.br:2119/jobmanager-lcgpbs-prod - FreeCPUs 239 - TotalCPUs 240 - CE: grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - CE grid001.cecalc.ula.ve:2119/jobmanager-lcgpbs-prod - FreeCPUs 22 - TotalCPUs 24 - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - FreeCPUs 522 - TotalCPUs 748 - CE: kuragua.uniandes.edu.co:2119/jobmanager-lcgpbs-prod - CE kuragua.uniandes.edu.co:2119/jobmanager-lcgpbs-prod - FreeCPUs 71 - TotalCPUs 84

6 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Get information from the Grid Find the sites who have shared home MPI directory –What happens when there is a MPI shared directory in the site?  The file is compiled in one WN, then the file read by the other WNs in the same site. –What happens when is not MPI shared directory?  The file is compiled in one WN, then the file must be copied to the others WNs. 6 [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPI_SHARED_HOME‘ - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday [dburbano@yali ~]$ lcg-info --vo prod.vo.eu-eela.eu --list-ce --query 'Tag=MPI_SHARED_HOME‘ - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-oneday - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-sixhour - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-thirtym - CE: gridgate.cs.tcd.ie:2119/jobmanager-pbs-threeday Tag=MPI-START

7 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Structure of a MPI job in the Grid with mpi-start 7 mpi-start-wrapper.sh mpi-Hooks.sh JDL File MPI file JobType = "Normal"; CPUNumber = 16; Executable = "mpi-start-wrapper.sh"; Arguments = "mpi-test MPICH2"; StdOutput = "mpi-test.out"; StdError = "mpi-test.err"; InputSandbox = {"mpi-start-wrapper.sh","mpi-hooks.sh","mpi-test.c"}; OutputSandbox = {"mpi-test.err","mpi-test.out"}; Requirements = Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment) && Member("MPICH2", other.GlueHostApplicationSoftwareRunTimeEnvironment); InputSandbox mpi-test.out mpi-test.err OutputSandbox mpi-start-wrapper.sh Sets up the environment for an specific MPI implementation mpi-hooks.sh Used before and after the execution of MPI program. - Pre-hook: Download data and compile MPI.c - Post-hook: Analyze and save data mpi.c MPI code

8 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Structure of a MPI job in the Grid with mpi-start 8 #!/bin/bash # Pull in the arguments. MY_EXECUTABLE=`pwd`/$1 echo "-------=======================---------------" pwd echo "-------=======================---------------" MPI_FLAVOR=$2 # Convert flavor to lowercase for passing to mpi-start. MPI_FLAVOR_LOWER=`echo $MPI_FLAVOR | tr '[:upper:]' '[:lower:]'` # Pull out the correct paths for the requested flavor. eval MPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH` # Ensure the prefix is correctly set. Don't rely on the defaults. eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH export I2G_${MPI_FLAVOR}_PREFIX # Touch the executable. It exist must for the shared file system check. # If it does not, then mpi-start may try to distribute the executable # when it shouldn't. touch $MY_EXECUTABLE # Setup for mpi-start. export I2G_MPI_APPLICATION=$MY_EXECUTABLE export I2G_MPI_APPLICATION_ARGS= export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER export I2G_MPI_PRE_RUN_HOOK=mpi-hooks.sh export I2G_MPI_POST_RUN_HOOK=mpi-hooks.sh # If these are set then you will get more debugging information. export I2G_MPI_START_VERBOSE=1 #export I2G_MPI_START_DEBUG=1 # Invoke mpi-start. $I2G_MPI_START JobType = "Normal"; CPUNumber = 16; Executable = "mpi-start-wrapper.sh"; Arguments = "mpi-test MPICH2"; StdOutput = "mpi-test.out"; StdError = "mpi-test.err"; InputSandbox = {"mpi-start-wrapper.sh","mpi-hooks.sh","mpi-test.c"}; OutputSandbox = {"mpi-test.err","mpi-test.out"}; Requirements = Member("MPI-START", other.GlueHostApplicationSoftwareRunTimeEnvironment) && Member("MPICH2", other.GlueHostApplicationSoftwareRunTimeEnvironment); Job.jdl mpi-start-wrapper.sh mpi-test MPICH2 $1 $2 MPI-start-wrapper.sh Environment variable in the system declared in the WN: /opt/i2g/bin/mpi-start Environment variables used inside of mpi-start script

9 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 9 #!/bin/bash # Pull in the arguments. MY_EXECUTABLE=`pwd`/$1 echo "-------=======================---------------" pwd echo "-------=======================---------------" MPI_FLAVOR=$2 # Convert flavor to lowercase for passing to mpi-start. MPI_FLAVOR_LOWER=`echo $MPI_FLAVOR | tr '[:upper:]' '[:lower:]'` # Pull out the correct paths for the requested flavor. eval MPI_PATH=`printenv MPI_${MPI_FLAVOR}_PATH` # Ensure the prefix is correctly set. Don't rely on the defaults. eval I2G_${MPI_FLAVOR}_PREFIX=$MPI_PATH export I2G_${MPI_FLAVOR}_PREFIX # Touch the executable. It exist must for the shared file system check. # If it does not, then mpi-start may try to distribute the executable # when it shouldn't. touch $MY_EXECUTABLE # Setup for mpi-start. export I2G_MPI_APPLICATION=$MY_EXECUTABLE export I2G_MPI_APPLICATION_ARGS= export I2G_MPI_TYPE=$MPI_FLAVOR_LOWER export I2G_MPI_PRE_RUN_HOOK=mpi-hooks.sh export I2G_MPI_POST_RUN_HOOK=mpi-hooks.sh # If these are set then you will get more debugging information. export I2G_MPI_START_VERBOSE=1 #export I2G_MPI_START_DEBUG=1 # Invoke mpi-start. $I2G_MPI_START #!/bin/sh # This function will be called before the MPI executable is started. # You can, for example, compile the executable itself. # pre_run_hook () { # Compile the program. echo "Compiling ${I2G_MPI_APPLICATION}" # Actually compile the program. echo "----------------------------------" pwd echo "----------------------------------" cmd="mpicc ${MPI_MPICC_OPTS} -o ${I2G_MPI_APPLICATION} ${I2G_MPI_APPLICATION}.c" echo $cmd $cmd if [ ! $? -eq 0 ]; then echo "Error compiling program. Exiting..." exit 1 fi # Everything's OK. echo "Successfully compiled ${I2G_MPI_APPLICATION}" return 0 } # This function will be called before the MPI executable is finished. # A typical case for this is to upload the results to a storage element. post_run_hook () { echo "----------------------------------" pwd echo "Executing post hook." echo "Finished the post hook." return 0 } MPI-start-wrapper.sh MPI-hooks.sh Compile the MPI C program Structure of a MPI job in the Grid with mpi-start

10 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 MPI Program 10 /* hello.c * Simple "Hello World" program in MPI. */ #include "mpi.h" #include int main(int argc, char *argv[]) { int numprocs; /* Number of process */ int procnum; /* process ID */ /* Initialize MPI */ MPI_Init(&argc, &argv); /* Find the ID of the process */ MPI_Comm_rank(MPI_COMM_WORLD, &procnum); /* Find the number of processors */ MPI_Comm_size(MPI_COMM_WORLD, &numprocs); printf ("Hello world! from processor %d out of %d\n", procnum, numprocs); /* Shut down MPI */ MPI_Finalize(); return 0; } MPI Code (Parallel code) Number of process that belong to the same set Process ID

11 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 Output job 11 -------=======================--------------- /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmis7XbLc1agm6w -------=======================--------------- ************************************************************************ UID = eelapr045 HOST = wn017.grid.cs.tcd.ie DATE = Wed Sep 30 14:53:38 IST 2009 VERSION = 0.0.59 ************************************************************************ mpi-start [INFO ]: search for scheduler mpi-start [INFO ]: activate support for pbs mpi-start [INFO ]: activate support for mpich2 mpi-start [INFO ]: call backend MPI implementation mpi-start [INFO ]: start program with mpirun - --------------------------------------------------- Compiling /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmis7XbLc1agm6w /mpi-test ---------------------------------- /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmis7XbLc1agm6w ---------------------------------- mpicc -m64 -o /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmis7XbLc1a gm6w/mpi-test /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmis7XbLc1a gm6w/mpi-test.c Successfully compiled /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmi s7XbLc1agm6w/mpi-test - ---------------------------------------------------- =[START]================================================================ Hello world! from processor 3 out of 4 Hello world! from processor 0 out of 4 Hello world! from processor 1 out of 4 Hello world! from processor 2 out of 4 =[FINISHED]============================================================= - --------------------------------------------------- ---------------------------------- /home/eelapr045/gram_scratch_yt24szh4sR/https_3a_2f_2fviani.uniandes.edu.co_3a9000_2fUHe6be5cmis7XbLc1agm6w ---------------------------------- Executing post hook. Finished the post hook. - ---------------------------------------------------- Mpi-test-out

12 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 12 #!/bin/sh -x # the binary to execute EXE=$1 echo "*********************************************" echo "Running on: $HOSTNAME" echo "As: " `whoami` echo "*********************************************" echo "Compiling binary: $EXE" echo mpicc -o ${EXE} ${EXE}.c mpicc -o ${EXE} ${EXE}.c echo "*************************************" if [ "x$PBS_NODEFILE" != "x" ] ; then echo "PBS Nodefile: $PBS_NODEFILE" HOST_NODEFILE=$PBS_NODEFILE fi if [ "x$LSB_HOSTS" != "x" ] ; then echo "LSF Hosts: $LSB_HOSTS" HOST_NODEFILE=`pwd`/lsf_nodefile.$$ for host in ${LSB_HOSTS} do echo $host >> ${HOST_NODEFILE} done fi if [ "x$HOST_NODEFILE" = "x" ]; then echo "No hosts file defined. Exiting..." exit fi echo "*************************************************" CPU_NEEDED=`cat $HOST_NODEFILE | wc -l` echo "Node count: $CPU_NEEDED" echo "Nodes in $HOST_NODEFILE: " cat $HOST_NODEFILE echo "************************************************" CPU_NEEDED=`cat $HOST_NODEFILE | wc -l` echo "Checking ssh for each node:" NODES=`cat $HOST_NODEFILE` for host in ${NODES} do echo "Checking $host..." ssh $host hostname Done echo "***********************************************" echo "Executing $EXE with mpiexec" chmod 755 $EXE mpiexec `pwd`/$EXE > mpiexec.out 2>&1 test-mpi.sh Type = "Job"; JobType = "Normal"; CPUNumber = 4; Executable = "test-mpi.sh"; Arguments = "test-mpi"; StdOutput = "test-mpi.out"; StdError = "test-mpi.err"; InputSandbox = {"test-mpi.sh","test-mpi.c"}; OutputSandbox = {"test-mpi.err","test-mpi.out","mpiexec.out"}; Requirements = Member("MPICH2", other.GlueHostApplicationSoftwareRunTimeEnvironment); Type = "Job"; JobType = "Normal"; CPUNumber = 4; Executable = "test-mpi.sh"; Arguments = "test-mpi"; StdOutput = "test-mpi.out"; StdError = "test-mpi.err"; InputSandbox = {"test-mpi.sh","test-mpi.c"}; OutputSandbox = {"test-mpi.err","test-mpi.out","mpiexec.out"}; Requirements = Member("MPICH2", other.GlueHostApplicationSoftwareRunTimeEnvironment); test-mpi.jdl Structure of a MPI job in the Grid without mpi-start

13 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 References Gridification FAQ: –http://applications.eu-eela.eu/grid_faq_MPICH.php?l=40&n=14 13

14 www.eu-eela.eu Querétaro (Mexico), E2GRIS2 28.09.2009 – 10.10.2009 14 Questions …


Download ppt "Www.eu-eela.org E-science grid facility for Europe and Latin America gLite MPI Tutorial for Grid School Daniel Alberto Burbano Sefair, Universidad de Los."

Similar presentations


Ads by Google