1. 1 High-Resolution Convective Modeling using WRF Coupled to NASA's Land Information System (LIS) Christa D. Peters-Lidard (PI) Hydrological Sciences Branch NASA/GSFC, Code 614.3 Christa.Peters@nasa.gov Christa.Peters@nasa.gov Wei-Kuo Tao (Co-I) Paul R. Houser (Co-I) Sujay V. Kumar Joseph L. Eastman Stephen E. Lang Yudong Tian Xiping Zeng

2. 2 Outline LIS Background LIS-WRF Coupling Design Computational Aspects Science Aspects Future Directions

3. 3 C. D. Peters-Lidard 1, P. R. Houser 1, S. V. Kumar 1, Y. Tian 1, J. Geiger 1, S. Olden 1, L. Lighty 1, J. L. Eastman 1, J. Sheffield 2, E. F. Wood 2, P. Dirmeyer 3, B. Doty 3, J. Adams 3, K. Mitchell 4, J. Meng 1,4, H. Wei 4 1 NASA, Goddard Space Flight Center Hydrological Sciences Branch, Code 974, Greenbelt, MD 2 Department of Civil and Environmental Engineering Princeton University, Princeton, NJ 3 Center for Ocean Land Atmosphere Studies (COLA) Calverton, MD 4 NCEP Environmental Modeling Center NOAA/NWS, Camp Springs, MD Code and Documentation at http://lis.gsfc.nasa.gov Background: LIS Team and Collaborators

4. 4 Background: LIS “Plugin” Design LIS driver domain-plugin lsm-plugin CLM Noah VIC Mosaic HySSIB SSIB Forcing- plugin Lat/lon Gaussian UTM GEOS GDAS AGRM NLDAS ECMWF CMAP Persiann Huffman CMORPH parameter-plugin soils LAI land cover elevation

5. 5 GrADS/DODS Server WRF GCE LSM Ensemble Noah, CLM2, Mosaic, HYSSiB, VIC LIS ESMF Coupled Uncoupled Global, Regional (Re-)Analyses or Forecasts Station Data Satellite Products Background: LIS Execution Modes ESMF

6. 6 LIS Background: Initial Benchmarking at NCEP Average Diurnal Cycles, March 2003 LIS with experimental mode of GFS on the gaussian horizontal grid of T62 Forcing: NCEP Global Reanalysis II, AGRMET radiation and CMAP precipitation. 30% more efficient in computing time CONUS AverageEurope Average Sensible Heat Ground Heat Latent Heat Net Radiation

7. 7 WRF-LIS: WRF Version http://www.wrf-model.org WRF V2.0 Release (May 18, 2004; V2.0.2 October, 2004, V2.0.3.1 December 2004) What is in WRF V2.0? Advanced Research WRF (ARW) dynamical core: Eulerian mass coordinate One-way and two-way nesting New physics options, including : Noah Land Surface Model (LSM), Rapid Update Cycle (RUC) LSM, Ysu Planetary Boundary Layer (PBL), and Grell-Devenyi ensemble cumulus scheme ESMF time manager Enhanced I/O options Enhanced Runtime System Library (RSL) New Standard Initialization (SI) V2.0 WRF 3-Dimensional Variational Assimilation (3DVAR) V2.0 Not actually ESMF, but a recoded F90 version of ESMF!! Key project requirement Built on Message Passing Interface

8. 8 Goddard Cumulus Ensemble (GCE) Model

9. 9

10. 10 The Goddard Cumulus Ensemble Model http://rsd.gsfc.nasa.gov/912/code912/model.html GCE V1.0 Release (June, 2004) What is in GCE V1.0? Place Land Surface Model (LSM) Message Passing Interface (MPI) parallelization Cyclic lateral boundary conditions GCE V2.0 Release (Expected July, 2005) What will be in GCE V2.0? Place and all LIS LSMs ESMF Virtual Machine-based MPI parallelism Cyclic and open boundary conditions 2D vs. 3D Key project requirement Project advancements Project advancement

11. 11 C. DeLuca/NCAR, J. Anderson/NCAR, V. Balaji/GFDL, B. Boville/NCAR, N. Collins/NCAR, T. Craig/NCAR, C. Cruz/GSFC, A. da Silva/GSFC, R. Hallberg/GFDL, C. Hill/MIT, M. Iredell/NCEP, R. Jacob/ANL, P. Jones/LANL, B. Kauffman/NCAR, J. Larson/ANL, J. Michalakes/NCAR, E. Schwab/NCAR, S. Smithline/GFDL, Q. Stout/U Mich, M. Suarez/GSFC, A. Trayanov/GSFC, S. Vasquez/NCAR, J. Wolfe/NCAR, W. Yang/NCEP, M. Young/NCEP and L. Zaslavsky/GSFC The Earth System Modeling Framework (ESMF) http://www.esmf.ucar.edu NSIPP Seasonal Forecast NCAR/LANL CCSM NCEP Forecast GFDL FMS Suite MITgcm NASA GSFC PSAS Climate Data Assimilation Weather

12. 12 ESMF Status http://www.esmf.ucar.edu Planned (Actual)Milestone May 2002Draft Developer’s Guide and Requirements Document completed 1 st Community Requirements Meeting and review held in D.C. July 2002ESMF VAlidation (EVA) suite assembled August 2002Architecture Document: major classes and their relationships Implementation Report: language strategy and programming model Software Build and Test Plan: sequencing and validation May 2003ESMF Version 1.0 release, 2 nd Community Meeting at GFDL November 2003First 3 interoperability experiments completed April 2004 (July 2004)Second API and Version 2.0 software release, 3 rd Community Meeting (Version 2.0.2 released in October 2004; and Version 2.1.0rp2 released in March 11, 2005) November 2004 (Expected Nov 2005) All interoperability experiments complete; all testbed applications compliant January 2005 (Expected Jan 2006) Final delivery of source code and documentation

13. 13 ESMF coupling schematic Low Level Utilities Fields and Grids Layer Model Layer Components Layer: Gridded Components Coupler Components External Libraries ESMF Infrastructure Model Component ESMF Superstructure BLAS, MPI, NetCDF, … ESMF Infrastructure Model Sub-component ESMF Superstructure BLAS, MPI, NetCDF, … Component Coupling: e.g., LIS-GCE Component Coupling: e.g., LIS-WRF ESMF Conceptual Design

14. 14 LIS-GCE and LIS-WRF coupling LIS-GCE LIS-WRF

15. 15 Evaluation Case Study: International H 2 O Project (IHOP), May-June 2002 Central US, Southern Great Plains

16. 16 IHOP “Golden Day” Synthetic Case GOES Imagery & Sounding Data June 6, 2002 13:55 UTC x GOES=Geostationary Operational Environmental Satellite

17. 17 Synthetic Case Experimental Design DryWetHalf Wet/Half Dry Bare SoilBaredryBarewetBarewetdry Vegetated (Grassland) GrassdryGrasswetGrasswetdry Soil Moisture Condition Vegetation Condition

18. 18 Comparison of Input Radiation Synthetic Case Evaluation

19. 19 Comparison of Energy Terms and Balance for Grasswet Case Synthetic Case Evaluation

20. 20 Factor Separation: Impact of Wet Soil and Grass Relative to Bare, Dry VariableWet SoilGrassInteraction GLW (Wm-2) Longwave Radiation -4.73.11.4 SWDOWN (Wm-2) Shortwave Radiation -2.04.3E-02-1.6 HFX (Wm-2) Sensible Heat Flux -15539-54 LH (Wm-2) Latent Heat Flux 176-5.8E-03116 Q2 (kg/kg) 2-m Air Humidity 3.82E-03-1.80E-041.54E-03 T2 (K) 2-m Air Temperature -5.490.26-0.72 PBLH Planetary Boundary Layer Height (m) -490191-149 Synthetic Case Evaluation

21. 21 Compute nodes used: 128 Impact of LSM on Uncoupled Performance

22. 22 Impact of ESMF on Coupled Performance Key conclusion: ESMF-compliant coupling adds minimal computational overhead relative to native models Weather Research and Forecasting Model (WRF) Goddard Cumulus Ensemble Model (GCE)

23. 23 WRF/LIS and WRF/GCE Performance scaling for the coupled systems

24. 24 Overview of LIS Spinup Impact on WRF+LIS Precipitation 24 hour accumulated precipitation with default soil initialization 24 hour accumulated precipitation with LIS 7.5 year spinup soil initialization Scientific Evaluation: June 12 “Real” Case

25. 25 Radar Derived Precipitation vs. Modeled Precipitation (mm) 6GMT June13th, 2002 Observed Radar Derived Surface Precipitation (Source: NOAA/NCEP) Modeled WRF+LIS Precipitation using LIS initial surface conditions Scientific Evaluation: June 12 Case

26. 26 Scientific Evaluation: June 12 Case Observed: Radar+Gauge WRF/LIS Modeled using Radar+Gauge precipitation for LIS uncoupled Spinup WRF/LIS Modeled using NOAA Model precipitation (GDAS) for LIS uncoupled Spinup WRF/LIS Modeled using WRF Standard Initialiation (WRFSI) w/o LIS spinup GDAS=Global Data Assimilation System (NOAA/National Centers for Environmental Prediction (NCEP))

27. 27 Project Highlights and Future Plans 1.ESMF-Compliant Coupling of WRF-LIS LISv4.0 has been coupled to WRFv.2.0.3.1 using ESMF v.2.1.0rp2 and a subcomponent coupling design Coupling needs to be generalized to handle: multiple nests, projections and grids; multiple PBLs; and LSMs other than Noah and CLM2. 2.Recognition LIS selected as co-winner of NASA 2005 Software of the Year Award 3.Benchmarking on Columbia We have been allocated 150,000 hours on NASA’s SGI Altix Supercomputer Columbia for benchmarking and additional scaling studies for the LIS/WRF and LIS/GCE coupled systems.