1. ROMS/TOMS Numerical Algorithms Hernan G. Arango IMCS, Rutgers University New Brunswick, NJ, USA

2. Developers and Collaborators Hernan G. Arango Alexander F. Shchepetkin W. Paul Budgell Bruce D. Cornuelle Emanuele DiLorenzo Tal Ezer Mark Hadfield Kate Hedstrom Robert Hetland John Klinck Arthur J. Miller Andrew M. Moore Christopher Sherwood Rich Signell John C. Warner John Wilkin Rutgers University UCLA IMR, Norway SIO Princeton University NIWA, New Zealand University of Alaska, ARSC TAMU Old Dominion SIO University of Colorado USGS/WHOI SACLAND USGS/WHOI Rutgers University

3. (Relief Image from NOAA Animation by Rutgers) Our models are used in oceanographic studies in over 30 countries by: Universities Government Agencies Companies Totaling 280 registered users on six continents

4. Free-surface, hydrostatic, primitive equation model Generalized, terrain-following vertical coordinates Boundary-fitted, orthogonal curvilinear, horizontal coordinates on an Arakawa C-grid Non-homogeneous time-stepping algorithm Accurate discretization of the baroclinic pressure gradient term High-order advection schemes Continuous, monotonic reconstruction of vertical gradients to maintain high-order accuracy KERNEL ATTRIBUTES

5. Vertical Terrain-following Coordinates Dubrovnik (Croatia) Vieste (Italy) Longitude Depth (m)

6. Curvilinear Transformation

7. Model Grid Configuration Nested Composed

8. ROMS/TOMS GOVERNING EQUATIONS

9. Momentum Governing Equations

10. Tracers Governing Equations

11. Continuity Equation Vertical Velocity

12. Deviatoric Transverse Stress Tensor

13. Parabolic Splines Reconstruction

14. Dispersive Properties of Advection  /2  /4 3  /4 kxkx 1/2 1 3/2 2 5/2 K(k)  x 2 4 6 10 8 Parabolic Splines Vs Finite Centered Differences

15. Barotropic Filters

16. Barotropic Power-Law Shape Functions

18. Barotropic-Baroclinic Coupling

19. ROMS/TOMS: MODULAR DESIGN

20. Modular, efficient, and portable Fortran code (F90/ F95) C-preprocessing managing Multiple levels of nesting and composed grids Lateral boundary conditions options for closed, periodic, and radiation Arbitrary number of tracers (active and passive) Input and output NetCDF data structure Support for parallel execution on both shared- and distributed -memory architectures CODE DESIGN

32. Coarse-grained parallelization PARALLEL DESIGN

33. } } Nx Ny PARALLEL TILE PARTITIONS 8 x 8

34. Coarse-grained parallelization PARALLEL DESIGN Shared-memory, compiler depend directives MAIN (OpenMP standard) Distributed-memory (MPI; SMS) Optimized for cache-bound computers ZIG-ZAG cycling sequence of tile partitions Few synchronization points (around 6) Serial and Parallel I/O (via NetCDF) Efficiency 4-64 threads

35. Horizontal mixing of tracers along level, geopotential, isopycnic surfaces Transverse, isotropic stress tensor for momentum General Length-Scale turbulence closure (GOTM) Local, Mellor-Yamada, level 2.5, closure scheme Non-local, K-profile, surface and bottom closure scheme SUBGRID-SCALE PARAMETERIZATION

36. Air-Sea interaction boundary layer from COARE (Fairall et al., 1996) Oceanic surface boundary layer (KPP; Large et al., 1994) Oceanic bottom boundary layer (inverted KPP; Durski, 2001) BOUNDARY LAYERS

38. Air-Sea interaction boundary layer from COARE (Fairall et al., 1996) Oceanic surface boundary layer (KPP; Large et al., 1994) Oceanic bottom boundary layer (inverted KPP; Durski et al., 2001) BOUNDARY LAYERS Wave / Current / Sediment bed boundary layer (Styles and Glenn, 2000)

39. Vertical Mixing and Sediment Models

40. Turbulence Sub-Models (parameterization of = -n t du/dz) Zero equation models – Prescribe n t One-equation models – n t ~ kl –Equation for k (advection/diffusion/production/dissipation) –Prescription for l Two-equation models –Equation for k –Equation for l Higher-order closures

41. MY25K-EK-W p0.0 3.0-1.0 m1.0 1.5 0.5 n1.0 -1.0-1.0 Umlauf and Burchard (2002) Generic Length Scale Turbulence Model Eq. 1: Eq. 2:

42. Suspended sediment transport model Transport Erosion Deposition - consolidation - bioturbation Internal bed dynamics Water Sediment Bed 1 2 3 4 5 6

43. Constant bed slope Freshwater flow at east boundary Tidal elevation and vertical salinity gradient at west boundary Four turbulence models Estuary Test

44. Estuary Test: Salinity Front Warner and Sherwood (USGS) MY 2.5 GLS - KKL GLS - KE GLS - KW

45. Estuary Test: Suspended Sediment Warner and Sherwood (USGS) MY 2.5 GLS - KKL GLS - KE GLS - KW

46. Lagrangian Drifters (Klinck, Hadfield) Tidal Forcing (Hetland, Signell) River Runoff (Hetland, Signell, Geyer) Sediment erosion, transport and deposition (Warner, Sherwood, Blaas) Sea-Ice (Budgell, Hedstrom) Biology Fasham-type Model (Moisan, Di Lorenzo, Shchepetkin) EcoSim Bio-Optical Model (Bissett) MODULES

47. Models predict that surface particles travel offshore and bottom particles travel onshore during upwelling events Surface and sub-surface drifters (Rutgers-LEO) Red: Surface Blue: Bottom metersmeters

48. (PMEL/NOAA) Ocean currents transport fish larvae along the continental shelf in the Gulf of Alaska The Coastal Gulf of Alaska

49. Gulf of Maine M2 Tides Surface Elevation (m) 20 km Resolution (Hetland,Signell)

50. Tidal currents around Martha's Vineyard and Nantucket (Rutgers) Temperature (color) and currents (arrows)

51. 5 10 1520 25 Distance (km) -10 -15 -25 -20 -5 Depth (m) 30 25 20 15 10 5 Salinity (PSS) Hudson River Estuary Resolution 10 m (Hetland)

52. Southern California Bight (1950-1998) SIO/UCSD

53. ROMS/TOMS: Tangent Linear and Adjoint Model Transformations

54. Tangent Linear and Adjoint Models There are many uses for tangent linear and adjoint models. The adjoint operator is a general sensitivity operator since it yields gradient information. In fact, any minimization or maximization that involves the linearized dynamical operators of a system will usually yield the adjoint equation. Thus, adjoint models are very useful for finding extrema.

55. Some uses of Adjoint Models 1.Data Assimilation Provides cost function gradient information. Can be used to fit model solutions to data by adjusting initial conditions, boundary conditions and parameters. 2.Sensitivity Analysis Sensitivity of a model solution to variations in model parameters can be evaluated very efficiently using adjoint models. 3.Eigenmode Analysis The eigenmodes of the tangent linear equations represent dynamic modes of variability (i.e. “normal modes”). These are our primary motivations for developing the tangent linear and adjoint versions of ROMS:

56. Some uses of Adjoint Models … 4.Adjoint Eigenmode Analysis: The eigenmodes of the adjoint equations represent the optimal excitations of the corresponding eigenmodes with respect to the chosen form. Most dynamical systems of interest are non-normal => Eigenmodes and adjoint eigenmodes are distinct. 5.Singular Vectors: These are the most rapidly growing perturbations that exist (linear limit) for the dynamical system with respect to chosen norm. Very useful for assessing the stability of the system. Very useful for perturbing the model initial conditions for generating ensembles of forecasts.

57. Some uses of Adjoint Models … 6.Stochastic Optimals (SOs) and Forcing Singular Vectors (FSVs): The most disruptive patterns of forcing for the TL model (with respect to a chosen norm). SOs assume forcing with zero time mean and stochastic in time. FSVs assume forcing that is time invariant. SOs and FSVs are useful for generating ensembles of forecasts. 7.Pseudospectra: Non-normal systems can display enhanced sensitivity and a large response to forcing frequencies that are far removed from resonant eigenmode frequencies. Pseudospectra are computed as the maximum singular values of the TL system resolvent.

58. The tangent linear model used in 1 - 7 above is generally a different tangent linear model to that used in the IOM representer method. Two Tangent Linear Models, Same Adjoint

59. Consider the 1D advection equation: This is the tangent linear model for perturbations δ u that is generally used in 1-7. (1) Let Therefore,

60. (2) Alternatively, This is the tangent linear model used in the IOM representer Formulation.

61. FINAL REMARKS

62. Open Source Code Pro Non-exclusive Complete information transfer More users: more tests, wider range of applications, more tools,… More developers: more bug fixes, more processes, better algorithms,… Con More difficult to manage Requires volunteers Risk of misuse Intellectual property issues

63. Source Codes Pre- and Post-procesing software Forum, Chat Room, News, and Documentation SERVICES http://www.ocean-modeling.org/ http://marine.rutgers.edu/po/models/roms/index.php

64. Publications Ezer, T., H.G. Arango and A.F. Shchepetkin, 2002: Developments in Terrain-Following Ocean Models: Intercomparisons of Numerical Aspects, Ocean Modelling, 4, 249-267. Haidvogel, D.B., H.G. Arango, K. Hedstrom, A. Beckmann, P. Malanotte-Rizzoli, and A.F. Shchepetkin, 2000: Model Evaluation Experiments in the North Atlantic: Simulations in Nonlinear Terrain-Following Coordinates, Dyn. Atmos. Oceans, 32, 239-281. MacCready, P. and W.R. Geyer, 2001: Estuarine Salt Flux through an Isoline Surface, J. Geoph. Res., 106, 11629-11639. Malanotte-Rizzoli, P., K. Hedstrom, H.G. Arango, and D.B. Haidvogel, 2000: Water Mass Pathways Between the Subtropical and Tropical Ocean in a Climatological Simulation of the North Atlantic Ocean Circulation, Dyn. Atmos. Oceans, 32, 331-371. Marchesiello, P., J.C. McWilliams and A.F. Shchepetkin, 2003: Equilibriom Structure and Dynamics of the California Current System, J. Phys. Oceanogr., 34, 1-37. Marchesiello, P., J.C. McWilliams, and A.F. Shchepetkin, 2001: Open Boundary Conditions for Long-Term Integration of Regional Ocean Models, Ocean Modelling, 3, 1-20. Moore, A.M., H.G. Arango, A.J. Miller, B.D. Cornuelle, E. Di Lorenzo, and D.J. Neilson, 2003: A Comprehensive Ocean Prediction and Analysis System Based on the Tangent Linear and Adjoint Components of a Regional Ocean Model, Ocean Modelling, Submitted. Peven, P., C. Roy, A. Colin de Verdiere and J. Largier, 2000: Simulation and Quantification of a Coastal Jet Retention Process Using a Barotropic Model, Oceanol. Acta, 23, 615-634. Peven, P., J.R.E. Lutjeharms, P. Marchesiello, C. Roy and S.J. Weeks, 2001: Generation of Cyclonic Eddies by the Agulhas Current in the Lee of the Agulhas Bank, Geophys. Res. Let., 27, 1055-1058. Shchepetkin, A.F. and J.C. McWilliams, 2003: The Regional Ocean Modeling System: A Split-Explicit, Free-Surface Topography-Following Coordinates Ocean Model, J. Comp. Phys., Submitted. Shchepetkin, A.F. and J.C. McWilliams, 2003: A Method for Computing Horizontal Pressure-Gradient Force in an Oceanic Model with a Non-Aligned Vertical Coordinate, J. Geophys. Res., 108, 1-34. She, J. and J.M. Klinck, 2000: Flow Near Submarine Canyons Driven by Constant Winds, J. Geophys. Res., 105, 28671-28694. Warner, J.C., H.G. Arango, C. Sherwood, B. Butman, and Richard P. Signell, 2003: Implementation and Applications of a Generic Length Scale Turbulence Closure and Suspended Sediment Transport Algorithms into the 3D Oceanographic Model ROMS, Ocean Modelling, Submitted.