1. Parallel Systems Parallel Systems Tools Dr. Guy Tel-Zur

2. Agenda - I Environments – Where to execute my code? – Windows Native (DenioMPI, DevCPP, Visual Studio) Virtualization (Virtual Box and VMWare Player) PuTTY, WinSCP – Linux gcc ssh editors

3. Agenda - II Compiling and executing a MPI program Debugging (Allinea DDT) Profiling (Jumpshot3, IPM, Scalasca, OpenSpeed Shop) Eclipse PTP Torque

4. Environment - Infrastructure Code development – Your own laptop, lab computers Execution – The educational cluster

5. Deino MPI Installation The famous “cpi.c” Profiling http://mpi.deino.net/index.htm

6. Click here first

8. /* -*- Mode: C; c-basic-offset:4 ; -*- */ /* * (C) 2006 by Deino Software. * (C) 2001 by Argonne National Laboratory. * See COPYRIGHT in top-level directory. */ #include "mpi.h" #include double f(double); double f(double a) { return (4.0 / (1.0 + a*a)); } int main(int argc,char *argv[]) { int n, myid, numprocs, i; double PI25DT = 3.141592653589793238462643; double mypi, pi, h, sum, x; double startwtime = 0.0, endwtime; int namelen; char processor_name[MPI_MAX_PROCESSOR_NAME]; MPI_Init(&argc,&argv); MPI_Comm_size(MPI_COMM_WORLD,&numprocs); MPI_Comm_rank(MPI_COMM_WORLD,&myid); MPI_Get_processor_name(processor_name,&namelen); fprintf(stdout,"Process %d of %d is on %s\n", myid, numprocs, processor_name); fflush(stdout);

9. n = 10000;/* default # of rectangles */ if (myid == 0) startwtime = MPI_Wtime(); MPI_Bcast(&n, 1, MPI_INT, 0, MPI_COMM_WORLD); h = 1.0 / (double) n; sum = 0.0; /* A slightly better approach starts from large i and works back */ for (i = myid + 1; i <= n; i += numprocs) { x = h * ((double)i - 0.5); sum += f(x); } mypi = h * sum; MPI_Reduce(&mypi, &pi, 1, MPI_DOUBLE, MPI_SUM, 0, MPI_COMM_WORLD); if (myid == 0) { endwtime = MPI_Wtime(); printf("pi is approximately %.16f, Error is %.16f\n", pi, fabs(pi - PI25DT)); printf("wall clock time = %f\n", endwtime-startwtime); fflush(stdout); } MPI_Finalize(); return 0; }

14. Jumpshot: clog to slog conversion

15. Profiling

18. Code Development under Windows Open Source/Free – DevC++ – Cygwin – Virtualization Commercial – Visual Studio

19. DevC++ http://www.bloodshed.net/devcpp.html

20. Working with DevC++ Select the path to the installed MPI library

21. Adding the link to the MPI library

22. Compiler options - includes Select the path to the right include directory

23. Code Development under Linux Connecting from Windows to Linux – PuTTY (+ Xming) – Remote Desktop – VNC

24. PuTTY

26. Putty + X windows Install xming and xming fonts

27. Start first Xming Server

28. IPM Integrated Performance Monitoring Read instructions at: http://ipm-hpc.sourceforge.net/userguide.html Read instructions at: http://ipm-hpc.sourceforge.net/userguide.html

29. The Wrong Way and the Correct Way to Execute Jobs Batch Queuing System

30. Torque Frequently Used Shell CommandsBasic UsageExample qsubsubmit a pbs jobqsub [script]$ qsub job.pbs qstat show status of pbs batch jobs qstat [job_id]$ qstat 44 qdel delete pbs batch job qdel [job_id]$ qdel 44 qhold hold pbs batch jobs qhold [job_id]$ qhold 44 qrls release hold on pbs batch jobs qrls [job_id]$ qrls 44 Reference: http://www.rcc.uh.edu/hpc-docs/49-using-torque-to-submit-and-monitor- jobs.htmlhttp://www.rcc.uh.edu/hpc-docs/49-using-torque-to-submit-and-monitor- jobs.html Reference: http://www.rcc.uh.edu/hpc-docs/49-using-torque-to-submit-and-monitor- jobs.htmlhttp://www.rcc.uh.edu/hpc-docs/49-using-torque-to-submit-and-monitor- jobs.html

31. Torque Shell Commands to Check Queue and Job Status qstat -qlist all queues qstat -alist all jobs qstat -au userid list jobs for userid qstat -rlist running jobs qstat -f job_idlist full information about job_id qstat -Qf queuelist full information about queue qstat -B list summary status of the job server pbsnodeslist status of all compute nodes

32. Torque Basic #PBS commands #PBS -N myjobSet the job name #PBS -m aeMail status when the job completes #PBS -M your@email.addressMail to this address #PBS -l nodes=4Allocate specified number of nodes #PBS -l walltime=1:00:00 Inform the PBS scheduler of the expected runtime #PBS in your Job Script The job script is any shell script you normally run to execute your programs. The #PBS commands appear to be comments to the shell but when your script is submitted to the PBS job scheduler Next slide from: http://www.democritos.it/activities/IT- MC/documentation/newinterface/pages/runningcodes.htmlhttp://www.democritos.it/activities/IT- MC/documentation/newinterface/pages/runningcodes.html Next slide from: http://www.democritos.it/activities/IT- MC/documentation/newinterface/pages/runningcodes.htmlhttp://www.democritos.it/activities/IT- MC/documentation/newinterface/pages/runningcodes.html

33. OptionDescription #PBS -N myJobAssigns a job name. The default is the name of PBS job script. #PBS -l nodes=4:ppn=2The number of nodes and processors per node. #PBS -q queuenameAssigns the queue your job will use. #PBS -l walltime=01:00:00The maximum wall-clock time during which this job can run. #PBS -o mypath/my.outThe path and file name for standard output. #PBS -e mypath/my.errThe path and file name for standard error. #PBS -j oe Join option that merges the standard error stream with the standard output stream of the job. #PBS -W stagein=file_listCopies the file onto the execution host before the job starts. (*) #PBS -W stageout=file_listCopies the file from the execution host after the job completes. (*) #PBS -m bSends mail to the user when the job begins. #PBS -m eSends mail to the user when the job ends. #PBS -m aSends mail to the user when job aborts (with an error). #PBS -m ba Allows a user to have more than one command with the same flag by grouping the messages together on one line, else only the last command gets executed. #PBS -r nIndicates that a job should not rerun if it fails. #PBS -VExports all environment variables to the job.

34. Torque and MPI #!/bin/bash #! Name of the job ('MPI' in this example): #PBS -N MPI CMD="mpirun -tune -ppn $ppn -np $np $application $options“ #! Or this: #CMD="mpirun -np $np $application” eval $CMD #!/bin/bash #! Name of the job ('MPI' in this example): #PBS -N MPI CMD="mpirun -tune -ppn $ppn -np $np $application $options“ #! Or this: #CMD="mpirun -np $np $application” eval $CMD

35. Another example # The name of the script is myjob #PBS -N myjob # Only 1 hour wall-clock time will be given to this job #PBS -l walltime=1:00:00 # Number of cores to be allocated is 288. # always ask for complete nodes #PBS -l mppwidth=288 #PBS -e error_file.e #PBS -o output_file.o # Change to the work directory cd $PBS_O_WORKDIR # Run the executable named myexe # and write the output into my_output_file mpirun -n 288./myexe > my_output_file 2>&1 # The name of the script is myjob #PBS -N myjob # Only 1 hour wall-clock time will be given to this job #PBS -l walltime=1:00:00 # Number of cores to be allocated is 288. # always ask for complete nodes #PBS -l mppwidth=288 #PBS -e error_file.e #PBS -o output_file.o # Change to the work directory cd $PBS_O_WORKDIR # Run the executable named myexe # and write the output into my_output_file mpirun -n 288./myexe > my_output_file 2>&1

36. Parallel Debugger – Allinea DDT Not available at the BGU

37. Allinea DDT The Allinea DDT Parallel debugger Tutorials, WebinarsAllineaDDT Parallel debugger TutorialsWebinars The Allinea DDT Parallel debugger Tutorials, WebinarsAllineaDDT Parallel debugger TutorialsWebinars