Ancillary File Tutorial . Ancillary File Tutorial Keir Bovis June 2013 © Crown copyright Met Office

Overview of the tutorial What is an ancillary file ? Review and summarise some important ancillary files used in NWP. Creating ancillary files using the Central Ancillary Program (CAP) Creating ancillary files from PP formatted data and NetCDF. Note that the focus is NWP and not climate models although similar ancillaries are used in both © Crown copyright Met Office

What is an ancillary file ? Ancillary files are the mechanism by which external data sources are entered into the Unified Model (UM). Ancillaries files typically comprise static data held on disk in UM fields file format. Ancillary files may hold data relating to model orography, soil and vegetation types, and climatologies for sea surface temperature and sea ice amongst others. At the UK Met Office, the Central Ancillary Program (CAP) creates ancillary files by reading post-processed source data and writing them in UM fields file format. The next set of slides provide an overview of the types, content and data source of land-cover ancillary files. © Crown copyright Met Office

Ancillary files SST/ice climatology Assumed underpinning hierarchy Soil moisture/snow climatology Vegetation surface type and LAI Soil parameters Model orography (topography) These are the ancillary files we will be looking at. The different types of ancillary files have been assembled into a pyramid to illustrate the dependence between them. When they’ve all been created they can be used in a UM job. Land-sea mask © Crown copyright Met Office

Land-sea mask Name: qrparm.mask Key field: Application: Source: Land mask stored as a logical field (1=land, 0=sea) Application: Key ancillary used to differentiate between land and sea points. The field is usually stored in model start dumps. Source: International Geophysical Biosphere Programme (IGBP). http://edc2.usgs.gov/glcc/globe_int.php 1km resolution obtained from Advanced Very High Resolution Radiometer (AVHRR) data spanning April 1992 to March 1993. IGBP notes: Maps individual 1km grid points into 17 different classes that can be used to classify a point as land or sea.. evergreen needleleaf forest evergreen broadleaf forest deciduous needleleaf forest deciduous broadleaf forest mixed forest closed shrub open shrub woody savannah savannah grassland wetland cropland urban mosaic snow and ice barren water open sea missing data © Crown copyright Met Office

Land-sea mask Land points Sea points Note the absence of inland lakes. Majority of these are defined as land points with a surface type of inland water and not as sea points. © Crown copyright Met Office

Land-sea mask: post-processing The default land-sea mask generated by the Central Ancillary Program (CAP) may not be suitable for use in NWP models. For example, very small lakes and islands may lead to numerical instabilities and run-time failures in NWP models. Alternatively, some land points are missing in the IGBP data set, e.g. Hawaii, Chatham Islands, etc. A facility exists with the CAP to allow the user to override the default mask and reassign land to sea point and vice-versa. The overrides may be manually created using a facility in the CAP or by using the graphical IDL-based tool ‘edit_lsm’ (we will see both approaches in the workshop later). © Crown copyright Met Office

Model orography (topography) Name: qrparm.orog Key fields: Mean orography in grid box (metres above sea level), standard deviation and three gradient orography fields. Application: These fields are used in gravity wave drag scheme, radiation scheme and hydrological inundation models. Stored in model start dumps. Source: Global models use the GLOBE dataset at 1km resolution. http://www.ngdc.noaa.gov/mgg/topo/globe.html UK regional models use restricted military DTED dataset. Alternative data sets include the Shuttle Radar Topography Mission (SRTM) data set http://www2.jpl.nasa.gov/srtm/cbanddataproducts.html GLOBE data set notes: © Crown copyright Met Office

Model orography – mean field © Crown copyright Met Office

Orography: post-processing Filtering is applied to remove scales shorter than 6km so that features too small to generate flow blocking and gravity waves are not included. Inland lakes are automatically detected and given values as if they were land points. If a high lake such as Lake Victoria is given a height of zero then it is likely that the model will fail. © Crown copyright Met Office

Soil parameters Name: qrparm.soil Key fields: Application: Source: Soil moisture volume content at critical, wilting and saturation points. Application: Describe the physical properties of the soil and are important in correctly determining the surface energy / soil moisture balance and the exchange of moisture and heat between the atmosphere and land surface. Source: NWP previously used Wilson & Henderson-Sellers (WHS) at resolution 1° latitude X 1° longitude originally designed for climate models. More up-to-date Harmonised World Soils Database (HWSD) at 1km resolution is now used operationally. As quality is variable it is supplemented with other regional soils datasets using an optimal interpolation scheme. http://www.fao.org/nr/water/news/soil-db.html We explain the key fields on the next slide. © Crown copyright Met Office

Soil parameters – key soil fields Critical point After drainage has stopped, large pores in soil are full of air and smaller are full of water. Saturation point Soil pores filled with water and soil is said to be saturated. There is no air left in the soil. Wilting point The point at which a plant wilts, there is not enough water in the soil for the plant root to extract. Accurate specification of these fields is important as they are used in calculating the movement of water through the implemented soil hydraulic scheme The key fields we identified on the previous slide are described in a bit more detail on this slide. Saturation point: Plants need air and water in the soil. At saturation, no air is present and the plant will suffer. Many crops cannot withstand saturated soil conditions for a period of more than 2-5 days with the exception of some like rice.. Accurate specification of these fields is important as they are used in calculating the movement of water through the implemented soil hydraulic scheme. Their specification is dependant on the accuracy of the soils databases used, for example HWSD provides data for two soil levels; the top soil (0-30cm) and the subsoil (30-100cm) and for soil properties such as percentage sand, percentage silt, percentage clay, percentage organic carbon, volume percentage gravel. Irrigation Water Management: Introduction to irrigation, FAO. © Crown copyright Met Office

Soil parameters – soil hydraulics Soil hydraulic schemes are used to parameterise the rate of infiltration of water through the soil by describing the relationship between volumetric soil moisture and soil suction. Two different schemes have been implemented in Met Office models, Clapp and Hornberger and van Genuchten each requiring there own soil ancillaries. Correct parameterisation of each scheme will ensure correct water ponding and run-off over the ground surface. The key soil parameter fields are used to parameterise the soil hydraulic scheme. We have an example later in the soil moisture climatology ancillary © Crown copyright Met Office

Vegetation fraction Name: qrparm.veg.frac Key fields: Application: Fraction of surface type expressed as a value (0-1). Application: Describe the fraction of each grid box that occupies each surface type defined within the land-surface model. Used in land-surface model and surface analysis. Source: International Geophysical Biosphere Programme (IGBP). http://edc2.usgs.gov/glcc/globe_int.php © Crown copyright Met Office

Vegetation fraction types land-surface scheme implemented in the UM recognises 9 surface types: Broad leaf tree Needle leaf tree C3 grass C4 grass Shrub Urban Inland water Bare soil Land ice Correct specification of the surface type affects the surface-energy budget and the movement of water through transpiration. For example: © Crown copyright Met Office

Vegetation fraction types The impact of incorrect specification of surface types affects evapotranspiration in NWP models. Correct specification of the surface type affects the surface-energy budget and the movement of water through evapotranspiation. © Crown copyright Met Office

Veg fraction: post-processing Source IGBP data is interpolated onto the model grid and the fraction of each of the IGBP classes present is calculated. Not every IGBP grid point has a defined class and final totals are adjusted to remove any areas of missing data. IGBP classes consisting of less than 1% of the grid box are eliminated, the area reallocated to other classes. The fraction totals of the 9 UM surface types are calculated by mapping IGBP classes to UM surface types using mapping weights. © Crown copyright Met Office

Veg fraction: post-processing 17 IGBP vegetation classes Evergreen Needleleaf Forest Evergreen Broadleaf Forest Deciduous Needleleaf Forest Deciduous Broadleaf Forest Mixed Forest Closed Shrublands Open Shrublands Woody Savannas Savannas Grasslands Permanent Wetlands Croplands Urban and Built-Up Cropland/Natural Vegetation Mosaic Snow and Ice Barren or Sparsely Vegetated Water Bodies 9 fractional UM classes Fraction of broadleaf trees Fraction of needleleaf trees Fraction of C3 grass Fraction of C4 grass Fraction of shrub Fraction of urban Fraction of water Fraction of bare soil Fraction of ice are to be mapped on to Listed on the left are the 17 IGBP vegetation classes that we need to map onto the 9 vegetation fraction classes know in in the UM. To do this we use a mapping algorithm described on the next page. Most of the grasses divide into two physiological groups, using the C3 and C4 photosynthetic pathways for carbon fixation. C4 grasses have a photosynthetic pathway that particularly adapts them to hot climates and an atmosphere low in carbon dioxide. C3 grasses are referred to as "cool season grasses" while C4 plants are considered "warm season grasses". Grasses may be either annual or perennial. Annual Cool Season - wheat, rye, Annual Bluegrass (annual meadowgrass, Poa annua), and oat Perennial Cool Season - orchardgrass (cocksfoot, Dactylis glomerata), fescue (Festuca spp), Kentucky Bluegrass and perennial ryegrass (Lolium perenne) Annual Warm Season - corn, sudangrass, and pearl millet Perennial Warm Season - big bluestem, indiangrass, bermudagrass and switchgrass.[3] © Crown copyright Met Office

IGBP -> UM mapping weights for vegetation fraction totals UM surface types IGBP class Broadleaf Needleleaf C3 Grass C4 Grass Shrub Urban Water Bare soil Ice Evergreen needleleaf 0.0 70.0 20.0 10.0 Evergreen broadleaf 85.0 5.0 Deciduous needleleaf 65.0 25.0 Deciduous broadleaf 60.0 Mixed forest 35.0 Close shrub 15.0 Open shrub 50.0 Woody savanna Savanna 75.0 Grassland 66.0 15.7 4.9 13.5 Permanent wetland 80.0 Cropland 100.0 Cropland/natural mosaic 55.0 Snow and ice Barren Inland water Composite mapping This table shows the mapping weights of IGBP classes to UM surface types. Some IGBP classes map to many different UM surface types The relative weighted contribution to each UM surface type has is loosely based on Wilson and Henederson-sellers weightings. 1-to-1 mapping © Crown copyright Met Office

Leaf Area Index (LAI) Name: qrparm.veg.func Key field: Application: The Leaf Area Index (LAI) is a measure of leaf area on each of the vegetative surface types. Application: LAI used in land-surface model in the determination of surface moisture and heat fluxes, through impact on transpiration, controlling the partitioning between surface sensible and latent heat fluxes. Source: Seasonal climatology derived from monthly means of the Moderate Resolution Imaging Spectroradiometer (MODIS) global 4km LAI product, from 2005-2009. ftp://primavera.bu.edu/pub/datasets/MODIS/MOD15_BU/C5/LAI/data/monthly/4km/ © Crown copyright Met Office

Leaf Area Index (LAI) LAI is the total one-sided area of leaves in a vertical column, within a unit area of ground © Crown copyright Met Office

MODIS Broadleaf trees LAI July climatology Time-varying zonal mean Boreal forest growth December climatology Example to show how climatology captures forest growth. SH NH summer NH © Crown copyright Met Office

MODIS needleleaf trees LAI July climatology Time-varying zonal mean Boreal forest growth December climatology SH NH summer NH © Crown copyright Met Office

SST & sea-ice climatologies Name: qrclim.sst, qrclim.seaice Key fields: The skin SST is the temperature of the top few micrometres over sea points at 12 monthly intervals. Fraction of sea-ice in sea over at 12 monthly intervals. Application: Both fields used in land-surface model and in construction of surface analyses. Source: The Met Office Hadley Centre's sea-ice and Sea Surface Temperature (SST) data set, HadISST1, created from monthly globally-complete fields of SST and sea-ice concentration on a 1 degree latitude-longitude grid from 1870 to-date http://www-hc/~hadobs/www.hadobs.org/hadisst/ © Crown copyright Met Office

Sea surface temperature climatology Zonal mean of 12 month climatology Monthly climatology for July Example of NH summer sea warming. SH NH NH summer © Crown copyright Met Office

Sea-ice climatology Zonal mean of 12 month climatology Monthly climatology for January The sea ice thickness field is arbitrarily assigned values of 2m in the Arctic and 1m in the Antarctic. However, a separate dataset exists for models that have variable sea-ice thickness. © Crown copyright Met Office

Soil moisture climatology Name: qrclim.smow Key fields: The soil moisture content in each soil layer at 12 monthly intervals. Application: The soil moisture climatology is used during the creation of a soil moisture analysis. Different climatologies will exist depending on soil parameters and soil hydraulic scheme used. Source: The climatology is created by running the Joint UK Land Environment Simulator JULES model (standalone land-surface model) off-line for 10 years following a 6 year spin-up period. http://www.jchmr.org/jules/ © Crown copyright Met Office

Soil moisture climatology Created using WHS soil parameters Created using HWSD soil parameters We hope the new climatologies are more realistic in the climatologies derived from the HWSD soil parameters. For example we see drier response over sandy soils in Portugal and finer structures as a result of increased HWSD resolution. © Crown copyright Met Office

Snow climatology Name: qrclim.smow Key fields: Application: Source: The snow amount over land (kg/m-2) at 12 monthly intervals. Application: This ancillary is available for model initialisation. Source: Climatology created by constructed from Willmott and Rowe (WR) and source climatologies from the Atmospheric Model Intercomparison Project (AMIP) at 1° X 1° resolution. © Crown copyright Met Office

Snow climatology Zonal mean of 12 month climatology Monthly climatology for February SH NH NH winter © Crown copyright Met Office

Implementation of ancillary files We’ve looked at the different types of ancillary files used in the unified model and now we move onto examine how these files are stored and used. Specifically we will look at: How they are created using the CAP Other methods of creating them Briefly how to visualise them © Crown copyright Met Office

Creating ancillary files There are two methods used to create ancillary files: Create them using the Central Ancillary Program (CAP). Importing processed data into ancillary fieldsfile format from a PP (Post-processed) format A NetCDF format. We’ll look at the CAP first and then other methods of importing data. PP (Post-Processed) format is a file format developed at the Met Office for holfding data that is to be manipulated and visualised. A PP file does not conform to the UM file format. It has none of the headers records Each header of the PP file corrosponds to an entry in the lookup table in the UM file format. PP files are used for visualising and processing fieldsfile data in IDL or PV-Wave. Users may manipulate ancillary file data in PP format in IDL and then wish to convert it back to fieldsfile format for use in UM runs. © Crown copyright Met Office

The Central Ancillary Program (CAP) The CAP comprises a suite of FORTRAN, C and UNIX shell scripts that are run to construct ancillary files from a post-processed data source. Ancillary file content is generated according to a pre-determined list of fields implemented in the code. It is has been previously ported to IBM, NEC and Cray platforms. Can also run on LINUX for use in the workshop that follows. All code is version controlled using an subversion repository and FCM. © Crown copyright Met Office

pptoanc This UM utility may be used to create an UM ancillary format fieldsfile from a PP data source. PP files may have been used in IDL to carry out processing on ancillary file data. The PP file must be converted back to fieldsfile format before it can be used in the UM. It is run from the command line pptoanc -hpf -n <namelistFile> <PPinputFile> <ancillaryFile> An example namelist file is shown in the next slide. © Crown copyright Met Office

pptoanc namelist Number of field types, times and max levels in PP file. Mapping of UM stash codes to PP field codes. Number of levels for each code. Length of integer and real constants array (15 & 6 for ancillaries) &SIZES field_types=1, n_times=1, nlevels=70, n_pp_files=1, field_code=287, stash_code=57, nlevs_code=70, len_intc=15, len_realc=6 / &LOGICALS single_time=.true. pack32=.true., pphead=.true. field_order=.true. lwfio=.true. &FIRST_VT fvhh=0,fvdd=0,fvmm=0,fvyy=0 &INTERVAL year360=f,ivhh=0,ivdd=0,ivdd=0,ivmm=0,ivyy=0 &LAST_VT lvhh=0,lvdd=0,lvmm=0,lvyy=0 &HEADER_DATA fixhd(2)=1 fixhd(3)=1 fixhd(12)=600 All input fields valid at a single time, pack real fields using 32 bit numbers print out pp headers read in, fields are ordered by time, output ancillary file or dump well-formed. Defines the first validity time of the ancillary data. Defines the time interval between the validity times. Defines the last validity time of the ancillary data. Used to overwrite values in the fieldsfile header: sub-model=atmosphere; vertical coordinate. type=hybrid; model version number x 100 + release. © Crown copyright Met Office

Xancil – creates ancillary files from NetCDF sources Written by Jeff Cole at the Department of Meteorology, University of Reading. Xancil takes NetCDF data and converts it into UM ancillary format. Comprises a front-end Tcl/Tk GUI with the same look and feel as xconv. Back-end processing is a compiled FORTRAN source single executable. Xancil contains none of the CAP functionality and cannot operate on un-processed source data for the purpose ancillary file creation. Now look at an alternative to converting NetCDF data into ancillary file format for use in the UM. © Crown copyright Met Office

Xancil overview Pre-defined set of known ancillary files to create Selected ancillary file has a pre-defined set of known fields Symbolic mapping to fields in NetCDF source Xancil works on a predefined set of known ancillary files Each ancillary file has a predefined set of know fields Xancil uses symbolic names to map NetCDF fields onto ancillary fields. © Crown copyright Met Office

Viewing ancillary files They can be viewed graphically using xconv or using in-house IDL routines such as pp_contour. Contents can be dumped using the UM ‘pumf’ (print UM file) utilities. Different versions of ancillary files can be compared using the UM ‘cumf’ (compare UM file) utility. © Crown copyright Met Office

Practical session By the end of the practical session you should be able to: Compile the CAP on a LINUX platform. Create a default land-sea mask for use in a global NWP model. Be able to manually edit the default land-sea mask to reassign land and sea points. Experiment in modifying a land-sea mask using the graphical edit_lsm tool. Please copy the pdf below to access the tutorial: /home/h01/frke/CAPworkshop.pdf (use evince to view it). Please ask if you get stuck or have any questions.. © Crown copyright Met Office

References Brutsaert,W. (1982). `Evaporation into the atmosphere. Theory, history and applications'. D. Reidel Publishing Company, Dordrecht. Calder,I.R., Wright,I.R. and Murdiyarso,D. (1986). `A study of evaporation from tropical rainforest ‑ West Java'. Journal of Hydrology, 89, pp13‑23. Cosby, B.J., Hornberger, G.M., Clapp, R.B. and Ginn, T.T. (1984). `A statistical exploration of the relationships of soil moisture characteristics to the physical properties of soils.` Water Resources Research, 20 pp682-690. Dickinson, A. and Wilson, C.W. (1991). `Interpolation Techniques and Grid Transformations used in the Unified Model'. (Unified Model Documentation Paper (UMDP) S1. Unpublished paper.) Dingman,S.L., Barry,R.G., Weller,G., Benson,C., Le Drew,E.F. and Goodwin,C.W. (1980). `Climate, snow cover, microclimate and hydrology'. In An Arctic ecosystem: The coastal tundra at Barrow Alaska, Brown,J., Miller,P.C., Tieszen,L.L. and Bunnel,F.L. (eds). Dowder, Hutchinson and Ross, Stroudsburg, © Crown copyright Met Office

References Eagleson (1970) `Dynamic Hydrology'. McGraw Hill, New York. Evans,R (1995) 'Creation of Data Fields Required for Effective Roughness Parameterisation'. Forecasting Research Technical Report No. 146. (Unpublished Paper). FAO (1995). ‘The digital soil map of the world’. FAO, Rome. Folland, C.K. and Parker, D.E. (1990). `Observed Variations of Sea Surface Temperature' in `Climate‑Ocean Interaction', M.E. Sclesinger (ed). Halldin,S., Saugier,B. and Pontailler,F.Y. (1984). `Evapotranspiration of adeciduous forest. Simulation using routine meteorological data'. Journal of Hydrology, 75, pp323‑341. Houldcroft,C.J., Grey,W.M.F., Barnsley,M., Taylor,C.M. and Los,S.O. New vegetation albedo parameters and global fields of backgroudn albedo derived from MODIS for use in climate model. To be published, currently available from Department of Geography, Swansea University, Singleton Park, Swansea, SA2 8PP. © Crown copyright Met Office

References Li,D. and Shine,K.P. (1995) ‘A 4-Dimensional Ozone Climatology for UGAMP Models’. UGAMP Internal Report No. 35 (Unpublished paper). Lindroth,A. (1985). `Canopy conductance of coniferous forests related to climate'. World Resources Research, 21, pp297‑304. Lu, S., Ren, T, Gong, Y and Horton,R (2007) An improved model for predicting soil thermal conductivity from water content at room temperature. Soil Science Society of America, 71, pp8-14. Mason, P.J. (1985). `On the parameterization of orographic drag', Proc. of the seminar on physical parametrizations for Numerical Models, Reading, 9‑13 September 1985, ECMWF. Meteorological Office (1991) `Meteorological Glossary`, HMSO. Monteith,J.L. (ed) (1976). `Vegetation and the Atmosphere, 2'. Academic Press, London. © Crown copyright Met Office

References Parker, D.E, Jackson, M., Horton, E.B. (1995) Forthcoming Climate Research Technical Note. Posey,J.W. and Clapp,P.F. (1964). `Global distributions of normal surface albedo'. Geofis. Int., 4, pp333 348. Peters-Lidard,C.D., Blackburn,E., Liang,X and Wood,E.F. (1998) The effect of soil thermal conductivity parameterization on surface energy fluxes and temperatures. Journal of the Atmospheric Sciences, 55, pp1209-1224. Post,W.M. Emanuel,W.R., Zinke,P.J. and Stangenberger,A.G. (1982). ‘Soil carbon pools and world life zones’. Nature 298 pp156-159. Robinson,D.A. and Kukla,G. (1984). `Albedo of a dissipating snow cover'. Journal of Climatology and Applied Meteorology, 23, pp1626 1634. Rockwood,A.A. and Cox,S.K. (1978). `Satellite infrared surface albedo over northwestern Africa'. Journal of Atmospheric Sciences, 35, pp513 522. © Crown copyright Met Office

References Scholes,R.J., Skole,D. and Ingram,J.S. (1995). ‘A global database of soil properties: Proposal for implementation. IGBP-DIS working paper #10, University of Paris, France. Shuttleworth,W.J., Gash,J., Lloyd,C.R., Moore,C.J., Roberts,J., Marques Filo,A.O., Silvia Filo,V.P., Ribeiro,M.N.G., Molion,L.C.B., De Sa,D.A., Nobre,J.C.A., Cabral,O.M.R., Patel,S.R. and De Moraes,J.C. (1984). `Eddy correlation measurements of energy partition for the Amazonian forest'. Quarterly Journal of the Royal Meteorological Society, 110, pp1143 1162. Sud,Y.C. and Smith,W.E. (1984). `Ensemble formulation of surface fluxes and improvement in evapotranspiration and cloud parameterizations in a GCM'. Boundary Layer Meteorology, 29, pp185 210. Thompson,N., Barrie,I.A. and Ayles,M. (1981) . `The Meteorological Office rainfall and evaporation calculation system: MORECS. Hydrological Memorandum No. 45, Meteorological Office, Bracknell. © Crown copyright Met Office

References Van Genuchten, M.Th. (1980). ‘A closed form of the equation for predicting the hydraulic conductivity of unsaturated soils. Soil Sci Soc. Am. J., 44, pp892-898. Warrilow, D.A. and Buckley, E. (1989). `The impact of land surface processes on the moisture balance of a climate model'. Annales Geophysicae, 7, (5) pp439‑450. Warrilow,D.A., Sangster,A.B. and Slingo,A. (1986). `Modelling of land surface processes and their influence on European climate'. Meteorological Office, Dynamical Climatology technical note 38, (unpublished paper). Willmott, C.J., Rowe, C.M. and Mintz, Y. (1985). `Climatology of the Terrestrial Seasonal Water Cycle', Journal of Climatology, 5, pp589‑606. Wilson, M.F. and Henderson‑Sellers, A. (1985). `A global archive of land cover and soils data for use in general circulation climate models'. Journal of Climatology, 5, pp119‑143. © Crown copyright Met Office

References Woodward,F.I., Smith,T.M. and Emanuel,W.R. (1995). ‘A global land primary productivity and phytogeograpy model.’ Global Biogeochemical Cycles, 9, pp471-490. Woodward, S. (2001). ‘Modeling the atmospheric life cycle and radiative impact of mineral dust in the Hadley Centre climate model.’ Journal of Geophysical Research, 106, No D16, pp18155-18166. Zinke,P.J., Stangenberger,A.G., Post,W.M., Emanuel,W.R. and Olson,J.S. (1986). ‘Worldwide organic soil carbon and nitrogen data’. NDP-018, Carbon Dioxide Information Center, Oak Ridge National Laboratory, Oak Ridge, Tennessee, USA. © Crown copyright Met Office