Building CCSM2.0 Brian Kauffman CCSM Software Engineering Group

Building the CCSM - overview The CCSM directory structure The build environment The execution directory Input data positioning Building executables interactively These slides follow the “Building the CCSM” section of the “CCSM2.0 User’s Guide”

The CCSM Directory Structure scripts/ contains one or more sets of run scripts for various simulations models/ contains the CCSM2.0 src code used in the various simulations

…/ccsm2.0/scripts/ In the scripts/ directory we find… test.a1/- example run scripts provided tools/- utils used by run scripts gui_run/- script that runs the gui case.X1/- additional, user-created case.Y2/ run scripts

…/ccsm2.0/scripts/test.a1/ In the test.a1/ directory we find… test.a1…master run script atm.setup.csh…setup scripts for cpl.setup.csh component models *.setup.csh RESUBMIT…used for auto-resubmit ccsm_archive…output file archiver test.a1.har…output file harvester

$SCRIPTS The run script sets an env variable that specifies where the run and setup scripts are located. Eg. in test.a1.run we have… $SCRIPTS = /usr/jdoe/ccsm2.0/scripts/test.a1 Or, more accurately… $CSMROOT = /usr/jdoe/ccsm2.0 $CASE = test.a1 $SCRIPTS = $CSMROOT/scripts/$CASE

run script, env vars, setup scripts A user executes a master run script (eg. test.a1.run) which in turn calls several component model “setup scripts,” one for each component model The run script sets certain env variables that are sent to the setup scripts, much like input arguments in a subroutine call. These env variables are used to coordinate and control the behavior of the setup scripts. For example, $CSMBLD specifies a makefile location, thus allowing all component model setup scripts to use the same makefile!

…/ccsm2.0/models/ In the models/ directory we find… atm/- code for atmosphere models ice/, lnd/, ocn/- code for other models cpl/- code for the coupler bld/- makefile & related csm_share/- code shared by components camclm_share/- code shared by cam & clm utils/- code for utils, eg. esmf

…/ccsm2.0/models/atm/ In the models/atm/ directory we find the source code for three atmosphere models… cam/- source code for CAM datm5/- source code for datm5 latm5/- source code for latm5 Similarly, there are multiple ice, land, and ocean component models.

…/models/bld/ In the bld/ directory we find… Makefile…generic makefile Macros.AIX…machine specific Macros.IRIX64 macros definitions makdep.c…dependency generator This is the one makefile and dependency generator used to build all CCSM components. $CSMBLD = $CSMROOT/models/bld

…/bld/Makefile The Makefile is a machine-independent template -- machine-specific details are provide by a macro-definitions files. Eg. the Makefile contains generic commands like… $(FC) $(FFLAGS) $(INCDIR) $*.f90

…/bld/Macros.* Macros files specify a set of machine- dependent macros needed by the generic Makefile. Eg. Macros.AIX might define… FC = mpxlf90_r FFLAGS = -03 -qstrict -qtune=auto INCLDIR = -I/usr/lpp/ppe.poe/include …whereas Macros.IRIX64 will define this macros appropriately for an SGI O2K

…/bld/Macros. Macros files specify a set of machine- dependent macros. We provide macros files for several machines, but when it comes to your home-town supercomputer, you’re on your own! Eg. Macros. defines… FC = FFLAGS = INCLDIR = etc.

…/bld/makdep.c $CSMBLD also contains a dependency generator, makdep.c, this identifies common dependencies such as F90’s use module & include, and cpp #include This is extremely useful, especially for code development & hacking.

Filepath The Makefile uses a source code search path defined in a file called “Filepath” created by the setup scripts. For example, its contents might be… /usr/jdoe/ccsm2.0/scripts/test.a1/src.cpl /usr/jdoe/ccsm2.0/models/cpl/cpl5 /usr/jdoe/ccsm2.0/models/csm_share

Filepath & $VPATH  The Filepath file is used to define gnu-make’s $VPATH variable  The “current working directory” is always the first path in $VPATH  The makefile makes a list of all *.c, *.F90, and *.F files found in the search path -- all of these files will be compiled.  If more than one file with the same name exist, the first file found with that name will be compiled -- all others will be ignored.  The order of the paths in Filepath is the same order that gmake/$VPATH will use.

gmake So CCSM has a common build env, with a command line that looks like… unix> gmake -j 6 -f $CSMBLD/Makefile \ MACFILE=Macros.AIX \ VPFILE=Filepath \ THREAD = TRUE \ EXEC=$EXEDIR/cpl

$EXEDIR Atmosphere component model is built here… $EXEROOT/atm/obj/*.o $EXEROOT/atm/obj/Filepath …but executable will be located one level up… $EXEROOT/atm/atm Similar for all other component models. Note, for example, $EXEROOT = /scratch/jdoe/test.a1

Input data positioning A component model setup script… Prepositions input data files Creates a namelist input file Builds the executable … all for one specific component. The setup script not only builds an executable, it also pre-positions data into the same directory in which the executable is located.

$CSMROOT/scripts/toos/ccsm_getdata The ccsm_getdata (csh script)

fini