ORNL is managed by UT-Battelle for the US Department of Energy Titan Cross Compile Adam Simpson OLCF User Support

2 Cross Compiling Titan node structure - connectivity External Login Service Nodes Internet SSH aprun qsub Compute Nodes

3 Cross Compiling Titan node structure - processor External Login –4 x 8 core AMD Magny-Cours Service Nodes –6 core AMD Istanbul Compute Nodes –16 core AMD Interlagos

4 Cross Compiling Cray Compile Wrapper CC, cc, ftn Controlled by xt-asyncpe module Sets compiler based on PrgEnv-* module –Same command for PGI, GNU, Cray, Intel Sets optimizations and target processor/arch. Adds libraries based on loaded Cray modules –Networking/Communication libraries(MPI, GNI, PMI, etc.) –Science Libraries(CUDA, LibSci, NetCDF,...) –ALPS libraries

5 Cross Compiling Compile for login/service nodes Don’t run wrapper compiled code on non compute nodes –Illegal instruction Interlagos has AVX/SSE4 instructions –Missing shared libraries Error while loading shared libraries: libalpslli.so.0

6 Cross Compiling Compile for login/service nodes module load craype-target-local_host –Compile for external login / service node* –Doesn’t add Cray specific libraries –Doesn’t work as advertised module load craype-target-native –Target node common instruction set –Doesn’t add any module related libraries No MPI, LibSci, CUDA, etc. Bypass wrappers –pgcc, pgCC, pgfortran : gcc, g++, gfortran –craycc, crayc++, crayftn : icc, icpc, ifort

7 Cross Compiling Compile for login/service nodes If Cray libraries required on external logins – module switch craype-interlagos craype-mc8 If Cray libraries required on service nodes – module switch craype-interlagos craype-istanbul

8 Cross Compiling CLE 5.2 Changes xt-asyncpe : deprecated –renamed craype craype-target-* : deprecated –craype-interlagos –craype-istanbul –craype-mc8 – craype-network-gemini – craype-network-none