CCSM4 Tony Craig NCAR Dec 5, 2008

Topics CCSM4 Overview CCSM4 Usage Adding new components Status of CCSM4 Development

Subversion setenv SVNREPO setenv SVNDATA You can browse these on the web svn co $SVNREPO/ccsm4/tags/ccsm4_0_alpha38 –Look for “Subversion Issues”, “Introduction to Subversion” Google subversion

CCSM4 Directory Structure ccsm4_0_alpha38 ChangeLogmodelsscripts atmlnd ocnicedrvcsm_shareutils popdomxocnsocndocn7seq_mctshreshrdshr piomctesmf_tmtimingpilgrimdriver seq_ccsm_drv.F90

Inputdata for common shared input datasets that are likely to be reused; grids, external forcing, initial files, etc local area is /wrkdir/maslowsk/RACM/inputdata subject to scrubbing permanent area is /archive/u2/wes/maslowsk/RACM Using these areas –directories should be rwxrwxr-x –files should be r--r--r-- –use unique filenames (date stamped) –never modify or remove ANY inputdata files some automated scripts for backup and recovery from Gabriele

Model Machine Layout (processor layout set in env_mach_pes) Driver lndice cpl atm ocn time processors

Coupling Issues Grids –global vs regional –consistent masks/fractions –staggering/averaging Frequency –tight coupling of atm/lnd/ice (hourly+) –loose coupling of ocn (daily) Consistency of coupling frequency, model timestep Conservation, flux accumulation and averaging Fields Flux calculations Albedo/shortwave consistency Runoff

Topics CCSM4 Overview CCSM4 Usage Adding new components Status of CCSM4 Development

Compsets, Grids, Machines Note: Compsets and Grids have a short name and a long name Compsets are the combination of components used –X = all dead models –A = all data models –B = all active models –others are C, D, F, G, H, I Grids define the resolution –standard resolutions have lnd grid = atm grid, ice grid = ocean grid (f19_g15) –recently added “trigrid” support, atm and land grids different (f45T42_g35) Machine is, for example, midnight > cd scripts >./create_newcase -list

create_newcase (1) > cd scripts > cat README >./create_newcase -help must specify case name, compset, resolution, and machine. these cannot be changed inside the case directory. always use -skip_rundb option >./create_newcase -case Btest -res f45_g35 -compset B -mach midnight - skip_rundb a case called Btest will be generated in the scripts directory the -case argument can be dir/name

create_newcase (2) > cd Btest edit env files –env files are mixture of xml and csh –env_mach_pes sets the pe layout. this must be done before configure is run. –env_run sets run length, restart frequency, etc –set LOGDIR to $CASEROOT/logs in env_mach_generic –set DOUT_S to FALSE in env_run.xml > configure -case configure generates build and run scripts plus the Buildnml_Prestage and Buildexe directories some env files are locked after configure. to reset, type “configure - cleanall”

create_newcase (3) >./Btest.midnight.build > qsub Btest.midnight.run The case will build under $EXEROOT, $EXEROOT is set in env_mach_generic.xml The case will run under $EXEROOT/ccsm_se Things I haven’t described: –setting up a full production run –setting the pe layout –archiving –how the scripts work –Buildnml_Prestage and Buildexe directories

Output files LID is a date string, yymmdd-hhmmss created by CCSM4 everytime it runs to generate unique filenames log files, *.log.$LID –each component has own log file. stdout/stderr are redirected to ccsm.log.$LID. –look in $EXEROOT/ccsm_se –look in $LOGDIR if set –look in short term archive area if DOUT_S is TRUE timing files –directory in $CASEROOT/timing –ccsm_timing_summary.$LID has performance summary history and restart files –look in $EXEROOT/ccsm_se –look in short term archive area if DOUT_S is TRUE

create_test (1) > cd scripts >./create_test -help must specify testname; $test.$resolution.$compset.$machine recommend using -testid >./create_test -testname ERS.f45_g35.B.midnight -testid t001 create_test automatically configure the test, designed to require no user intervention. will create a test named ERS.f45_g35.B.midnight.t001 -testroot option allows the testcase to be place in another directory generate and compare are supported for regression testing

create_test (2) > cd ERS.f45_g35.B.midnight.t001 >./ERS.f45_g35.B.midnight.t001.build > qsub ERS.f45_g35.B.midnight.t001.test Test results are in the file TestStatus The file TestStatus.out provide more test result details log files should be copied into the test directory under a directory called logs

Topics CCSM4 Overview CCSM4 Usage Adding new components Status of CCSM4 Development

*_comp_mct.F90 (1) The *_comp_mct.F90 provides the coupling between the driver and the component. This is the top level of any component Driver looks like call lnd_init_mct() while (.not. stop_alarm) call seq_timemgr_clockAdvance() call lnd_run_mct() end do call lnd_final_mct() Must have an init, run, and finalize method

*_comp_mct.F90 (2) Interface generally contains –ESMF_Clock –seq_cdata datatype that holds mct_gGrid (domain information) mct_gsMap (decomposition information) seq_infodata_type (scalar data) MPI Comm, Component ID, etc –Attribute vectors (2d fields), one for data to the component, one for data from the component. Use an existing version and modify it for your component

init_mct method enter extract mpicom using seq_cdata_setptrs. this is THE mpi communicator for your component. get ccsm4 settings using seq_infodata_GetData() initialize and synchronize your component clock using seq_timemgr_EClockGetData() set the global seg map which describes the grid decomposition of your component. set the global grid which prescribes grid field data of your component. allocate the attribute vectors to be used to pass data back and forth. pack the export attribute vector with initial data set some ccsm4 settings using seq_infodata_PutData() return NOTE: there are literally dozens of examples as each component has implemented this. Review and Reuse that code.

run_mct method enter check ccsm4 settings using seq_infodata_GetData() get clock information using seq_timemgr_EClockGetData() unpack the import attribute vector “run your component” pack the export attribute vector set some ccsm4 settings using seq_infodata_PutData() verify the internal clock is in sync with the CCSM4 clock return

Coupling Interface Final Notes The driver will hand you an mpi communicator. You must use this inside your component. The components must check that they are staying in sync with the driver clock. The component must respond to a restart flag sent from the driver. The albedo/shortwave interaction between components is complicated and critical. This will probably cause you problems at some point.

A few other issues Unit numbers History files Namelist input Mapping files for coupler

Component Template Scripts Under each component is a bld directory and a script called *.cpl7.template. This script is what the CCSM4 scripts execute to generate component specific scripts under your $CASEROOT Buildnml_Prestage and Buildexe directories. New components will have to implement a version of this for their component.

Topics CCSM4 Overview CCSM4 Usage Adding new components Status of CCSM4 Development

Status of CCSM4 development beta series New science GLC model scripts continue to evolve