1. 7/9/20161 cms.ncas.ac.uk March 2014 NCAS Unified Model Introduction Part 1: Overview of the UM system York – March 2014

2. March 2014 2 What is the UM? UM User Interface (UMUI) Running the UM Contents

3. March 2014 3 The Unified Model is…  owned and developed by UK Met Office, used for both commercial weather forecasts as well as climate research and advice to government, for example IPCC reports  PUM (Ported UM) is used under licence by UK academic community for weather and climate research  used under licence by forecast/climate agencies around the world [Norway, South Africa, Australia…]  The UM is used by a large and diverse community of users.  The UM is expected to run on a diverse range of computers.

4. March 2014 4 At the UK Met Office Unified Model code Atmosphere Ocean Data assimilation system UKMO Data archive Observations UMUI SCSCAP

5. March 2014 5 External UM Unified Model code Atmosphere Ocean UMUI Contains 1)UM code + fixes 2)UM User guide + documentation + code browser 3)UM Tools 4)Installation examples + 2 or 3 example jobs tested on 2 or 3 computers+compilers

6. March 2014 6 NCAS supported UM on HPC services (HECToR, MONSooN) Unified Model code Atmosphere Ocean UMUI Contains 1)UM code installed, optimised, tested and fixed 2)Changes to UMUI and archiving mechanism 3)Tools and utilities both from UKMO and NCAS 4)Managed data from Met Office and ECMWF + many user jobs, both climate and NWP HPC Data archive

7. March 2014 7 UM Versions The Met Office (UKMO) release new UM versions regularly (approx every 3 months). Each version is independent and stands alone. NCAS may not always port each new version to HECToR or MONSooN NCAS has very limited effort to port new versions to external systems NCAS carries out limited testing and provides only sample jobs for users Old Dynamics New Dynamics Rose UKMO 4.5 4.7 6.1 6.6.3 7.3 8.0 8.2 8.4 8.5 8.6 8.5 9.x NCAS: ARCHER (N)y Y Y Y…………………………. MONSooN Y Y Y Y…………………………. Code Maintenance nupdate pumscm FCM Ocean Code old old old NEMO

8. March 2014 8 Unified Model (UM) versions 4.5 6.17.3 configurations Hadcm3 Hadgam1 Hadgem3-AO experiments Nugam IPCC scenario A3  The UM is not a single fixed piece of code.  In the UK academic community there is a broad UM spectrum as multiple UM versions are used new UM configurations are developed a large variety of UM experiments are performed QESM PS18 6.6.3 Hadgem2-ES 7.6

9. March 2014 9 The Unified Model (UM) is a collection of numerical modelling software which … Can be used in atmosphere only, ocean only, or coupled or with a slab ocean Can be coupled to other ocean models via OASIS e.g. Hadopa, or other components NEMO, CICE, JULES (for example in HadGEM3) May be run in many different modes  aquaplanet  global or mesoscale  regional coupling

10. March 2014 10 runs at different resolutions  Global climate (N48, N96, N144, N216, N512)  LAM (Limited Area Model) (UK 12 km, 4 km)  L38 - L60 – increased vertical resolution  Higem – N144 atmosphere + 1/3 ⁰ ocean N48N96N144N216 270 km135 km90 km60 km 96 x 73192 x 145288 x 217432 x 325 3.75º x 2.4º1.88º x 1.24º1.25º x 0.83º0.83º x 0.56º

11. March 2014 11 What is the UM? UM User Interface (UMUI) Running the UM

12. March 2014 12 UMUI – UM User Interface The UMUI… is a windows interface to the UM is written in Tcl/Tk is currently offered via the PUMA service – which provides all the latest UMUI changes – centralised access to local files (hand-edits etc) – common job database for ALL academic users creates the scripts which control the UM run (some changes to these scripts may be needed for new bits of code – hand-edits) has been extended to accommodate non-UKMO users assigns values to namelist switches used by the UM at runtime selects code branches used to build the model (UM6.6.3+)

13. March 2014 13 UMUI – creating UM jobs Choose a UM job which is close to the one you want to run either from a colleague, UKMO or from NCAS - either copy the job into your own job if they are in the same local database - or upload a basis file, the full description of a UM job which has come from another UMUI database, into your own job NCAS standard jobs should be available under the userid umui Basis files can be downloaded from the UMUI and emailed.

14. March 2014 14 UMUI - vocabulary Experiments : xxab - a grouping of UM jobs - identified by a description in the UMUI - only your experiments are displayed by the UMUI, unless otherwise requested - experiments can be created, copied and deleted Jobs : xxabc - up to 26 jobs can be grouped in an experiment - jobs can be created, copied and deleted - 2 jobs can be differenced Navigation windows input windows

15. March 2014 15

16. March 2014 16 UM – Submitting the model UMUI scripts namelists code MONSooN HECToR submit What do you need to know about the system where the UM is installed? directory structure and disk space setup job submission mechanism and queue structures input files available e.g. branches, start files, ancillary files The UM uses namelists for setting parameters at runtime, these are set in the UMUI.

17. March 2014 17 What is the UM? UM User Interface (UMUI) Running the UM

18. March 2014 18 Running a UM job: Stage 1 - Compilation UMUI – what options? Compilation and Run Options -> Compile and run options for Atmos & Recon Options : compile only, compile and run, run from a named executable Compile Overrides: Used on some computers for compiler options to speed up some sections of the code or to overcome problems. FCM manages code components creates makefiles for compilation and loading compiles and links the code according to options selected creates an executable

19. March 2014 19 Running a UM job: Stage 2 - Reconfiguration Reconfiguration is a standalone program which modifies (reconfigures) UM atmosphere or ocean start files FCM code management compiles the code runs the reconfiguration in parallel for new dynamics in the serial queue for the old dynamics produces new start files UMUI – what options? Compilation and Run Options -> Compile and run options for Atmos & Recon Options: run from an existing executable, or compile and create an executable

20. March 2014 20 Running a UM job: Stage 3 - Running UMUI – what options? Input/Output Control & Resources -> Start date and run length options For length of model run and the number of processors to use you need to understand the computer’s queue structure and the performance of your job. User Information & Submit Method -> Job submission method qsub for Cray XE6 LoadLeveller for IBMs @ for PCs and workstations

21. March 2014 21 STASH Output Filenames Job Output UM Output

22. March 2014 22 STASH Spatial and Temporal Averaging and Storage Handling 1) Select diagnostic from Load New Diagnostic section 2) Time profilewhen diagnostic will be output (start and end time and frequency) whether time processing required (accumulation, mean, time series) 3) Domain profile vertical (specify which levels) horizontal (limited area, meaning [ zonal, vertical, meridional, horizontal], weighting) 4) Usage profile select output unit for the diagnostic

23. March 2014 23

24. March 2014 24 STASH in the UMUI The UMUI windows for STASH are different from model windows. Available diagnostics are organised in sections. Some diagnostics have HELP information, a lot don’t! Diagnostics -> load new diagnostics -> (double click on the section) Just because a diagnostic is available doesn’t mean it works!

25. March 2014 25 Verify diagnostics (Ctrl+V) Package switched onPackage switched off

26. March 2014 26 UM - STASHmaster file H1| SUBMODEL_NUMBER=1 H2| SUBMODEL_NAME=ATMOS H3| UM_VERSION=8.4 # #|Model |Sectn | Item |Name | #|Space |Point | Time | Grid |LevelT|LevelF|LevelL|PseudT|PseudF|PseudL|LevCom| #| Option Codes | Version Mask | Halo | #|DataT |DumpP | PC1 PC2 PC3 PC4 PC5 PC6 PC7 PC8 PC9 PCA | #|Rotate| PPF | USER | LBVC | BLEV | TLEV |RBLEVV| CFLL | CFFF | # #=============================================================================== # 1| 1 | 0 | 2 |U COMPNT OF WIND AFTER TIMESTEP | 2| 2 | 0 | 1 | 18 | 1 | 1 | 2 | 0 | 0 | 0 | 0 | 3| 000000000000010000000000000000 | 00000000000000000001 | 1 | 4| 1 | 2 | -3 -3 -3 -3 -12 20 -99 -99 -99 -99 | 5| 0 | 56 | 0 | 65 | 0 | 0 | 0 | 0 | 5 | # Contains all the information on all the atmosphere diagnostics available

27. March 2014 27 STASH - Adding new diagnostics Users can add new diagnostics Advice via the CMS website (STASH User Guide) User must provide a STASHmaster file local to the UMUI Users can copy diagnostics settings from one UM job to another using: copy_stash This script takes as input the two UM basis files. The two jobs must be the same UM version.

28. March 2014 Output listing from the UM 28 Output listing quite large and confusing Output directory $HOME/um/umui_out Output listing filename, e.g. xagmc000.xagmc.o98342.t14136.leave Output listing options controlled by UMUI Input/Output Control & Resources -> Output Choices Check output listing for - timings reported at the end of the listing file - key words like ERROR, ABORT, "file not found" Check presence of key files, executable after compilation start file after reconfiguration output files after model run

29. March 2014 29 SUMMARY

30. March 2014 30 How a UM experiment (HADAM3, UKMES, UK4km) is defined. A standard named UM experiment has a defined version of the UM (4.5, 6.1) a particular horizontal and vertical resolution particular settings in the UMUI + hand-edits, which change the settings in the UMUI include a controlled list of ‘mods’, which are changes to the UM source code or particular branches in FCM use a defined set of input files, start files, ancillary files and lateral boundary conditions => Named experiments can change and evolve at a different rate to UM versions

31. March 2014 NCAS Unified Model Introduction Part 2: Code Maintenance

32. March 2014 32 … is different for different versions Version 4.5 nupdate (either original Cray version or UKMO C version) Version 6.1 pumscm (almost the same as nupdate written in perl) Version 6.3 and above FCM (Flexible Configuration Management) - written in perl - based on subversion UM Code Maintenance…

33. March 2014 UM 4.5 and 6.1 33 UM User Interface (UMUI) PUMA UM code library Input files Output Analysis Analysis and Visualisation Local 1) Setting up the UM job 2) Running the UM job 3) Analysis of UM Output The UMUI produces a set of scripts to run the UM that assumes that the UM has been installed with the structure created by the installation procedure. Super Computer

34. March 2014 34 UM 4.5/6.1 - Code UM code is written in Fortran with some C Fortran 77 with Fortran 90 extensions as well as Fortran 90 uses NUPDATE (and derivatives) as a code maintenance tool - either former Cray version now ported to IBM, Linux, HP, DEC, Sun, SGI - or UMKO portable version with limited functionality - or PUMSCM (in perl) for the New Dynamics versions NUPDATE user guide available on the NCAS-CMS web site, PUMSCM has nearly the same functionality but with some extra features.

35. March 2014 35 UM 4.5/6.1 - NUPDATE/PUMSCM DECK FRED DECK JOE Use a modification set (modset or mod) to change the program library. *IDENT TESTP */ test print statement *DECLARE FRED *I FRED.24 print *, ‘ temp in CONVEC’, T Program Library each line in the program library has a unique identifier. a new program library is only created for new versions of the UM

36. March 2014 36 UM 4.5/6.1 - NUPDATE/PUMSCM Program Library (for each version of the UM) modset (change or modification of the code) nupdate or pumscm *.f (compile file for each subroutine or deck) defs (nupdate controlled definitions) apply

37. March 2014 37 From UM 6.6.3 onwards changes to the Unified Model source code is managed using: FCM (Flexible Configuration Management) source code managed through subversion full history of source code change using trac and wiki code extract system managed through scripts build system with scripts and make UM 6.6.3 onwards - Code

38. March 2014 38 UMUI UM code Input files Output Analysis Analysis and Visualisation Local subversion +trac+wiki User logs in User UM code HPC service PUMA NCAS UM FCM service is only available on PUMA. UM6.6.3 onwards