What is ESMF and what does it mean to adopt it? 3 rd ESMF Community Meeting Cecelia DeLuca Nancy Collins Jon Wolfe Climate Data Assimilation Weather GMAO Seasonal Forecast NCAR/LANL CCSM NCEP Forecast GFDL FMS Suite MITgcm NASA GMAO Analysis

15 July 2004 Outline What is ESMF? ESMF users A closer look at ESMF Adoption strategies – A couple suggestions for how to get started

15 July 2004 ESMF Architecture 1.ESMF provides an environment for assembling geophysical components into applications, with support for ensembles and hierarchies. 2.ESMF provides a toolkit that components use to i.increase interoperability ii.improve performance portability iii.abstract common services

15 July 2004 Hierarchies and Ensembles assim sea ice ocean land atm physicsdycore assim_atm atmland Seasonal Forecast coupler ESMF encourages applications to be assembled hierarchically Coupling interfaces are standard at each layer Components can be used in different contexts assim_atm Ensemble Forecast assim_atm ESMF supports ensembles with multiple instances of components running sequentially (and soon, concurrently)

15 July 2004 ESMF Data Classes Model data is contained in a hierarchy of multi-use classes. The user can reference a Fortran array to an Array or Field, or retrieve a Fortran array out of an Array or Field. Array – holds a Fortran array (with other info, such as halo size) Field – holds an Array, an associated Grid, and metadata Bundle – collection of Fields on the same Grid State – contains States, Bundles, Fields, and/or Arrays Component – associated with an Import and Export State

15 July 2004 ESMF Utilities Communication libraries Regridding library (parallelized, on-line SCRIP) Message logging Calendar management – clocks, alarms, calendars Configuration (replaces namelists) IO – currently netCDF Performance profilng (not implemented yet)

15 July 2004 Outline What is ESMF? ESMF users A closer look at ESMF Adoption strategies – A couple suggestions for how to get started

15 July 2004 What kind of user are you? Individual/Small Group Large Group/Institution User Provider Support routines, utilities, communication code, leverage code developed elsewhere Full framework, inter-platform, inter-institution solutions Provide components to others without large overhead Package code to run many places under many systems, support, documentation

15 July 2004 What do you get from ESMF? Different running configurations – Standalone – Coupled to data only (dead components) – Coupled to live components – Strategy for structuring complicated applications Different communication strategies Different data decompositions Utilities

15 July 2004 What do you get (cont.) Standard component interfaces Coupler-building utilities

15 July 2004 Outline What is ESMF? ESMF users A closer look at ESMF Adoption strategies – A couple suggestions for how to get started

15 July 2004 ESMF Superstructure Classes Gridded component – Models, data assimilation systems - “real code” Coupler component – Data transformations between Gridded Components State - Data sent between Components AppDriver – Generic driver

15 July 2004 ESMF Components “Your code goes here” Framework specifies interface, user supplies code which is to be called Initialize/Run/Finalize subroutine paradigm Can create nested subcomponents

15 July 2004 ESMF Gridded Components Does the computational work Depending on how the current code is structured, may be possible to wrap without structural changes Or might use ESMF conversion for excuse to make structural changes!

15 July 2004 ESMF Gridded Components (cont) States for import/export Sequential mode (concurrent coming soon) – All import data available at start – All export data ready at end Registration routine (SetServices) to tell the framework what routines you’re supplying

15 July 2004 ESMF Coupler Components States for import/export Does the transformation needed to take data from one Component and make it fit the requirements of another Component Not automatic - needs to be customized for each new configuration Expected to be thin, however - making use of the transformation routines in ESMF

15 July 2004 ESMF States Description/reference to other ESMF data objects All data passed between Components is in the form of States and States only Data is referenced so does not need to be duplicated Can be Bundles, Fields, Arrays, States, or name-placeholders

15 July 2004 ESMF Infrastructure Data Classes Grids – Coordinates and grid decomposition Arrays – Hold data Fields – include Grid, Array, and metadata Bundles – Fields packed together for convenience, data locality, latency reduction

15 July 2004 ESMF DataMap Classes These classes give the user a systematic way of expressing interleaving and memory layout, also hierarchically (partially implemented) ArrayDataMap – relation of array to decomposition and grid, row / column major order, complex type interleave FieldDataMap – interleave of vector components BundleDataMap – interleave of Fields in a Bundle

15 July 2004 ESMF TimeMgr Clocks - timesteps, time intervals Calendars - many different varieties Alarms - periodic, fixed time Can be used independent of other parts of ESMF

15 July 2004 ESMF Parallelism VM - hardware and OS abstraction DELayout - data decomposition Grids (ESMF_GridDistribute)

15 July 2004 Virtual Machine (VM) VM handles resource allocation Elements are Persistent Execution Threads or PETs PETs reflect the physical computer, and are one-to-one with Posix threads or MPI processes Parent Components assign PETs to child Components The VM communications layer does simpleMPI-like communications between PETs (alternative communication mechanisms are layered underneath)

15 July 2004 DELayout Handles decomposition Elements are Decomposition Elements, or DEs (decomposition that’s 2 pieces in x by 4 pieces in y is a 2 by 4 DELayout) DELayout maps DEs to PETs, can have more than one DE per PET (for cache blocking, user-managed OpenMP threading) Simple connectivity or more complex connectivity, with weights between DEs - users specify dimensions where greater connection speed is needed Array, Field, and Bundle methods perform inter-DE communications

15 July 2004 ESMF Communications Halo – Updates edge data for consistency between partitions Redistribution – No interpolation, only changes how the data is decomposed Regrid – Based on SCRIP – Methods include bilinear, conservative, higher-order conservative Bundle, Field, Array-level interfaces

15 July 2004 ESMF Quickstart Directory with the shell of an application 2 Gridded Components 1 Coupler Component 1 Top level Gridded Component 1 AppDriver main program

15 July 2004 ESMF Quickstart Application AppDriver (generic main program) Top level Gridded Component GridComp 1 GridComp 2 CplComp

15 July 2004 Outline What is ESMF? ESMF users A closer look at ESMF Adoption strategies – A couple suggestions for how to get started

15 July 2004 Adoption Strategies: Top Down Wrap existing application as ESMF Component Wrap existing data with Fields and put them into States Can run as a Component at this point Integrate Field level communication Use Time Manager, Config, LogErr

15 July 2004 Adoption Strategies: Bottom Up Use ESMF utilities as needed Incorporate the Time Manager, Config Use Grids, Arrays for internal data Use VM, DELayout routines for data decomposition Wrap data structures as Fields Use data communication routines like Halo, Regrid

15 July 2004 How can you learn more? Web site Examples Documentation Interoperability experiments Users forum

15 July 2004 What do you need to get started? Supported build platforms include SGI, IBM, Compaq, many Linuxes C++ (gcc ok) and Fortran 90/95 compiler MPI (or mpiuni substitute lib) Understand how to run a job

15 July 2004 Where do you go from here? Download ESMF 2.0 and compile it – or find someone who already has it on your platform of choice! Run the demo Run the quickstart application Look at the examples Start writing code!