1. To Couple or Not To Couple John Warner, USGS

2. Overview of some recent advancements to ROMS –sediment transport components –wave/current interactions –model coupling Focus on Model Coupling –MCT, ROMS, SWAN How to develop a coupled ROMS-SWAN application (inlet test example) Summary /where are we going? Outline

3. What is model coupling? Model A running on M nodes. 78 processor cluster - WHOI shared Model B running on N nodes. Model C ……… ……… (it also works here) …… with communications between all models.

4. Why Couple models? Your model doesn't have all the physics ! –For example, waves are needed from a different source, such as another model or data Provide dynamic feedback –nonlinear interactions between forcing (i.e. wave-currents-morphology). http://www.cofc.edu

5. Where / when is model coupling important ? Surface winds Groundwater Sea ice Waves http://www.cofc.edu/CGOInquiry/tidalinlets.htm

6. Model Coupling Toolkit Mathematics and Computer Science Division Argonne National Laboratory http://www-unix.mcs.anl.gov/mct/ R. Jacob, J. Larson, E. Ong, “M×N Communication and Parallel Interpolation in CCSM Using the Model Coupling Toolkit”, (Preprint) ANL/MCSP1225-0205, Mathematics and Computer Science Division, Argonne National Laboratory, Feb 2005. Submitted to International Journal for High Performance Computing Applications. J. Larson, R. Jacob, E. Ong, “The Model Coupling Toolkit: A New Fortran90 Toolkit for Building Multiphysics Parallel Coupled Models”, (Preprint) ANL/MCS-P1208-1204, Mathematics and Computer Science Division, Argonne National Laboratory, Dec 2004. Submitted to International Journal for High Performance Computing Applications. How are we coupling models? MCT is an open-source package that provides MPI based communications between all nodes of a distributed memory modeling component system. Download and compile as libraries that are linked to. (like netcdf) Model A "M" processors Model B "N" processors

7. Plan to change coupler to ESMF in the near future Earth System Modeling Framework http://www.esmf.ucar.edu/ Partners: NOAA Geophysical Fluid Dynamics Laboratory NOAA National Centers for Environmental Prediction NSF National Center for Atmospheric Research NASA Goddard Global Modeling and Assimilation Office NASA Goddard Institute for Space Studies NASA Jet Propulsion Laboratory NASA Goddard Land Information Systems project DOD Naval Research Laboratory DOD Air Force Weather Agency DOD Army Engineer Research and Development Center DOE Los Alamos National Laboratory DOE Argonne National Laboratory University of Michigan Princeton University Massachusetts Institute of TechnologyUCLA Center for Ocean-Land-Atmosphere Studies Programme for Integrated Earth System Modeling (PRISM) Common Component Architecture (CCA)

8. What do we need from SWAN, and what does SWAN want from ROMS? u, v, , h Dwave, Hwave, Lwave, Pwave_top, Pwave_bot, Ub_swan, Wave_dissip

9. Recent advancements to ROMS Roller Model

10. 3D momentum equations (wave - current interaction adds new terms) x: y: continuity: scalar transport: s:

11. Radiation stress terms wave energy: Mellor, G. L. (2003). The three-dimensional current and surface wave equations. Journal of Physical Oceanography 33, 1978-1989. where vertical shape functions: roller function: stokes velocities: horizontal radiation stresses: vertical radiation stresses: Roller model based on: Svendsen, I. A. (1984). Wave heights and set-up in a surf zone, Coastal Engineering, 8, 303-329. Svendsen, I, Haas, K and Zhao, Q (2002). Quasi-3D Nearshore Circulation Model SHORECIRC, CACR Report 2002-01. Mellor, G. L. (2005). Some consequences of the three-dimensional current and surface wave equations. Journal of Physical Oceanography 35, 2291-2298.

12. Depth-integrated equations group speed: x: y: continuity: horizontal depth-integrated radiation stresses: where stokes velocities: Velocities are Lagrangian !! So we are recoding to make them all Eulerian.

13. SWAN (the wave model) cx, cy = propagation velocities (x- and y- directions)  = relative frequency  = wave direction S = source/sink term for: - wind-wave generation - wave breaking - bottom dissipation - nonlinear wave-wave interactions SWAN accounts for shoaling, diffraction, partial transmission, and reflection. N = wave action density (energy density / relative frequency) Booij, N., R.C. Ris and L.H. Holthuijsen, 1999, A third-generation wave model for coastal regions, Part I, Model description and validation, J.Geoph.Research, 104, C4, 7649-7666. Booij, N., R.C. Ris and L.H. Holthuijsen, 1999, A third-generation wave model for coastal regions, Part II, Model description and validation, J.Geoph.Research, 104, C4, 7649-7666. Booij, N., Haagsma, IJ.G., Holthuijsen, L.H., Kieftenburg, A.T.M.M., Ris, R.C., van der Westhuysen, A.J., and Zijlema, M. (2004). SWAN Cycle III version 40.41 User Manual, Delft University of Technology. pretty good user’s manual !!!

14. Simple example of an uncoupled application to demonstrate wave driven flows ….

15. Shoreface test case test case setup 1) use SWAN to predict wave heights, wave dissip, wave direction, wave length, wave period 2) results from SWAN were used as forcing to ROMS and Shorecirc. Thanks to Kevin Haas at Ga Tech Savannah

16. How does the coupled modeling system work?

17. Model organization master.F mpi_init init run finalize init run finalize { { SWAN ROMS init_file (# procs/model)

18. init, run, and finalize ROMSSWAN init (grid decomp) run (sync. point) finalize init_param init_parallel init_scaclars init_coupling roms_init roms_run main3d..... waves_coupling... mpi_finalize close_io roms_finalize MPI_INIT SWINITMPI SWMAIN SWEXITMPI SWINIT SWREAD (grid) init_coupling swanmain..... ocean_coupling... mpi_finalize close_io

19. Grid decomposition (during initialization) ROMS SWAN -Each tile is on a separate processor. -Each tile registers with MCT.

20. init_coupling ROMS- init_coupling SWAN- init_coupling 1 1 2 2 3 3 processed by each ROMS tile processed by each SWAN tile

21. Synchronization (run phase) ROMS- cocean_oupling SWAN- waves_coupling MCT processed by each ROMS tileprocessed by each SWAN tile

22. How to set up a Coupled ROMS / SWAN application Use INLET_TEST as an example. This application is provided in ROMS 2.3 You need to compile and install MCT (only once). You need all the input files for a stand alone ROMS simulation. You need all the input files for a stand alone SWAN simulation. We have some m files to make ROMS grids, SWAN grids from ROMS grids, SWAN wave forcing files, SWAN wind forcing files, etc.

23. Step 1: cppdefs.h INLET_TEST is already set up to have model coupling NEARSHORE_MELLOR activates radiation stress terms in momentum equations. WAVES_OCEAN activates coupling to the wave model.

24. Step 2: coupling input file Set number of processors for each model. coupling_inlet_test.in Set synchronization interval. File names. (SWAN = INPUT)

25. Step 3: ROMS input file Make sure number of tiles = number of processors specified in coupling input file ocean_inlet_test.in

26. Step 4: SWAN input file Keep these 2 sets of commands for CURRENT and WLEV. These activate internal SWAN flags. INPUT MANY MORE commands further in this file. RTFSM.

27. Step 5: makefile SWAN_COUPLE activates compiler directives for MCT libraries. Needs MPI

28. Step 6: compile and run make mpirun –np M*N./oceanM ROMS/External/coupling_inlet_test.in

29. Inlet Test ubar = 0.5 m/s depth (m) 2 16 Hs = 2.0 m T = 10 s 1200 m Use the inlet test case as an example. This test case will be distributed with ROMS 2.3

30. Inlet test (preliminary look) 1) Two-way coupled simulation shows wave-current interaction: reduced magnitude of currents and increased wave height. 2) Coupled simulation has lower velocity and less erosion offshore of inlet. 3) Lateral BC effects of wave model. SWAN ROMS SWAN ROMS

31. Where are we heading with this?

32. New directory structure for ROMS 2.3 COAMPS WRF Others … (Natalie Perlin OSU, but not formalized into Rutgers version yet)

33. Summary Model enhancements include : –sediment transport components –wave/current interaction –model coupling Developed a coupled ocean-wave modeling system. New directory structure to allow coupling to other models. ROMS 2.3 release – in a few weeks after finalize Lagrangian to Eulerian issue.