1. www.bsc.es BSC tools hands-on session

2. 2 Objectives Copy ~nct00001/tools-material into your ${HOME} –cp –r ~nct00001/tools-material ${HOME} Contents of tools-material –bin/ Some BSC tools scripts to make your life easier –apps/ CGPOP mini-application for the hands-on session –slides/ All the slides wrt BSC tools –packages/ Paraver packages & tutorials –doc/ Extrae user guide

3. 3 Paraver Install the Paraver binaries in your laptop –Binaries for linux x86 & x86-64, windows & mac Configure the Paraver package –Set-up the tutorials directory Go into Help > Tutorials, click on Preferences Window –Edit Tutorials root to the directory that contains the tutorial »tools-material/packages/Tutorials

4. 4 Trace generation workflow Instrumentation MPIT Application Process Extrae Application Process Extrae Application Process Extrae lib MPIT Paraver Dimemas Clustering Folding … Analysis.prv.pcf.row

5. 5 Obtaining traces (CGPOP) Build the CGPOP application –cd ~/tools-material/apps/CGPOP –./build Generate trace-file for the CGPOP application –cd ~/tools-material/apps/CGPOP/jobscript –edit job.lsf Uncomment module load bsctools & TRACE definition –Optionally, choose a trace-file name –submit the job bsub < job.lsf

6. 6 Inside job.lsf #!/bin/bash #BSUB -n 48 #BSUB -oo cgpop_%J.out #BSUB -eo cgpop_%J.err #BSUB -R"span[ptile=16]" #BSUB -x # Exclusive use #BSUB -J cgpop #BSUB -W 00:10 #BSUB -U tools module load netcdf/4.1.3 # To trace, uncomment the two following lines and add TRACE # in between mpirun and the binary module load bsctools export TRACE=~/tools-material/apps/trace.mpi.f.sh export TRACE_NAME= time mpirun ${TRACE}../mpi2s1D/cgpop.linux_icc.180x120

7. 7 Inside trace.*.sh #!/bin/bash # Workaround for MN3 export TMPDIR=${TMPDIR}/extrae mkdir –p ${TMPDIR} export EXTRAE_CONFIG_FILE=${HOME}/tools-material/extrae/extrae.xml export LD_PRELOAD=${EXTRAE_HOME}/lib/libmpitrace.so $@ Execute the binary passed as a parameter Inject instrumentation code Configure instrumentation

8. 8 Library alternatives LibrarySerialMPIOpenMPpthread libseqtrace libmpitrace[f] 1 libomptrace libpttrace libompitrace[f] 1 libptmpitrace[f] 1 1 for Fortran codes

9. Extrae XML configuration PAPI_TOT_INS,PAPI_TOT_CYC,PAPI_L1_DCM,PAPI_L2_DCM,PAPI_L3_TCM,PAPI_FP_INS,PAPI_BR_MSP PAPI_TOT_INS,PAPI_TOT_CYC,PAPI_LD_INS,PAPI_SR_INS,RESOURCE_STALLS,PAPI_BR_UCN,PAPI_BR_CN,PAPI_VEC_ SP PAPI_TOT_INS,PAPI_TOT_CYC,RESOURCE_STALLS:LB,RESOURCE_STALLS:RS,RESOURCE_STALLS:SB,RESOURCE_S TALLS:ROB,PAPI_VEC_DP GUIA DE USUARIO explained.xml

10. Extrae XML configuration 500000 <merge enabled="yes" synchronization="default" tree-fan-out="16" max-memory="512" joint-states="yes" keep-mpits="yes" sort-addresses="yes" overwrite="yes" > $TRACE_NAME$

11. Extrae XML configuration 1-3 1-5

12. Using Paraver Analyse the results with Paraver Tutorials available –Tutorial 1: Introductory tutorial on the usage of the Paraver GUI –Tutorial 2: Analysis of HydroC application –Tutorial 3: Introductory tutorial on the usage of Dimemas –Tutorial 4: General Paraver & Dimemas analysis applied to WRF –Tutorial 5: General Paraver & Dimemas analysis methodology REMEMBER! Configuration files (*.cfg) can be applied to any trace-file.