1. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Numerical Weather Prediction Modeling for MURI/Atmospheric Parameter Retrievals John R. Mecikalski, Derek J. Posselt CIMSS Co-Investigators 1.Overview of NWP support 2.GIFTS Simulated Data for Algorithm & Product Development 3.Computational Requirements 4.Atmospheric Parameter Retrievals O U T L I N E

2. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin NWP Infrastructure at CIMSS PSU/NCAR MM5 UW-Nonhydrostatic Modeling System (UW-NMS) Weather Research & Forecasting (WRF) Rapid Update Cycle-2 (RUC2) The capabilities of numerically simulating the atmosphere over a wide range of meteorological scales, over large geographical domains, and for realtime numerical weather prediction (NWP) are rapidly increasing at CIMSS.

3. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin NWP support for Ongoing Projects Cloud-Radiative modeling for instrument validation, radiance and retrieval algorithm development (MURI PI & Co-I’s) Generate “truth” atmosphere for satellite- based estimates of PBL stability, and convective initiation studies (MURI Co-I’s & MURI PM) Assessment of turbulence with NAST-I data (MURI PM)

4. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin The UW-NMS and PSU/NCAR MM5 A robust NWP system: –scaleable –explicit physics at 4 km-resolution –explicit microphysics for accurate clouds –variably-stepped topography Excellent model for “Cloud-Radiative” experiments: Independent of other NWP systems Developed for multi-processor, distributed memory computational environments (Fortran-90 with MPI).

5. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin NWP for Retrieval & Simulation NWP can provide atmospheric retrieval algorithms critical “first guess” information. NWP for direct GIFTS data and instrument simulation: –A numerically simulated atmosphere is considered “nature” and is assumed to very accurately represent the true state. –Requires a sophisticated numerical model and is therefore computationally very expensive (1 Gflops per data “cube”)

6. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Improving Retrieval First Guess AERI (red) versus Radiosonde (black) First Guess Temperature Dew Point Temperature

7. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Altitude (km) April 1998-October 1999, 463 AERI+Model/ radiosonde profiles Differences are less than 1 deg K AERI+Model every 10 minutes, sondes every 3-12 hours AERI ETA MODEL AERI/MODEL - RADIOSONDE (K) NWP in Atmospheric Retrievals: AERI+Model/Radiosonde Temperature Comparison NWP MODEL GIFTS

8. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Simulating GIFTS Data High-resolution numerical simulations are used to provide the following atmospheric parameters to the GIFTS radiative-transfer model: –Temperature –Water vapor mixing ratio –Mixing ratios and mean particle diameters of cloud and ice liquid water –Liquid and ice water path –Cloud-top height with respect to both liquid and ice cloud Goal: To provide investigators with simulated (interferogram) data that accurately represents what will eventually come from the GIFTS instrument.

9. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Numerical Model Output to Simulated Radiances Simulated GIFTS Data Cube Simulated GIFTS Brightness Temperatures NWP Data Cube Clouds from UW-NMS Forward Model: Dave Tobin Forward Model: Dave Tobin

10. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Derived GIFTS Data Vertical Temperature Profiles 1 sounding per 1 scene pixel 128 x 128 = 16384 scene pixels 4 km pixel spatial resolution (nadir) “Regional Sounding” Product: 100 vertical layers Retrieved values/cube = 128x128x100 1.6 million retrieved values/cube 10 second dwell time Longitude (deg) Latitude (deg) Temperature (K) GIFTS SIMULATED TEMPERATURE DATA CUBE 950 MB LAYER TEMPERATURE GIFTS SIMULATED VERTICAL TEMPERATURE PROFILE Pressure (mb) Air Temperature (K)

11. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Modeling Infrastructure for Large Scale GIFTS-IOMI data processing Need to simulate GIFTS data at 4 x 4 km resolution over large domain: –Distributed memory, massively parallel computer code and computer system –Must be accomplished in a timely manner [O(few days)] –Demands a sophisticated atmospheric model (UW- NMS) Combination of UW-NMS and MM5 allows us to simulate “large” regional domains.

12. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Large-Scale Symmetric Multi-Processor (SMP) –Integrated unit with high-bandwidth backplane –Shared RAM, multiple CPU, single OS kernel –Communication: shared memory & semaphores –Examples: SGI Origin, IBM RS/6000 Linux Cluster –Network of inexpensive COTS computers –Multiple RAM, multiple CPU, multiple OS kernel –Communication: TCP/IP, sockets & datagrams –Examples: Sandia ‘CPlant’, Forecast Systems Laboratory ‘Jet’ 64-bit Linux Necessary Computational Infrastructure for MURI: SMP & Cluster Systems

13. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Hardware Diagram: Hardware Diagram: Linux Cluster Network Switch UPS Tape Archive Fileserver Computer Disk Array Rack and CPUs 70% of the initial cost30% of the initial cost

14. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Current Computational Limits 3 x 3 Cube Domain: 4 km resolution Number of grid points: 400 x 400 x 40 = 6400000 Approximate memory use: 15-20 Gb of RAM Total Cluster memory: 24 Gb of RAM

15. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Near-Term Limits 5 x 5 Cube Domain: 4 km resolution Number of grid points: 640 x 640 x 40 = 16384000 Anticipated memory use: 45-50 Gb of RAM 16 more processors (8 more nodes) 32-bit limit for domain set-up

16. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Future Needs 25 x 25 Cube Domain: 4 km resolution Number of grid points: 3200 x 3200 x 40 = 4.1x10 8 Approximate memory use: 1170 Gb of RAM Doable?

17. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Minimum Calculation Times for 3 by 3 Simulation (3 time steps) * Not parallelized yet

18. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Options for 25 x 25 Cubes Run a series of 3 x 3 cubes and concatenate them; use global model run to smooth edges SGI: How much RAM per processor? Itanium: How much RAM per processor? Waiting for compiler. Buy time on existing large cluster: Man-hours to spin up on such an effort. Wait for better MM5 or distributed memory UW-NMS Write distributed MM5 for domain set-up General Question: – Is 25 x 25 really necessary? – What are we going to do with data volume? – Winds?

19. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Atmospheric Parameter Retrievals Progress to Date: June 2002–May 2003 Temperature and Water Vapor – Jun Li First Winds – Chris Velden, Gail Dengel Stability Indices – Wayne Feltz, John Mecikalski Atmospheric (PBL) Turbulence – John Mecikalski, Ryan Torn, Wayne Feltz Visibility: “GVision” – Derek Posselt, Wayne Feltz, John Mecikalski, Tom Rink

20. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Atmospheric Temperature, Moisture, Ozone 1. The Physical retrieval model has been developed 2. Two fast ways for Jacobian calculation are developed 3. Contrast between surface skin temperature and surface air temperature on boundary layer moisture 4. Simulation studies using cube data from MM5 Clear Sky retrieval of T

21. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin First Winds Issues for Winds … Critical need for LARGE “cube” simulation data sets; without such large domains, useful wind sets from simulated GIFTS data are not possible – First “GIFTS” winds have been produced. – Issues of cube numbers when retrieving winds (2x2 or larger) – Issues of concatenating cubes

22. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Stability Indices “Truth” LCL Retrieved LCL First Retrieval of Stability Example: Lifted Condensation Level (LCL) LCL may be used as a measure of boundary layer depth, and/or the depth of the inversion atop the boundary layer (e.g., marine inversion). Higher Stability Lower Stability

23. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin PBL Turbulence Evaluating Turbulence from Hyperspectral Measurements Example: Shear-Driven Instabilities: AERI-derived Boundary Layer Depths CBL Waves & Rolls

24. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin PBL Turbulence June 23, 2001  autocorrelation starting 320 minutes after sunrise August 8, 2001  e autocorrelation starting 300 minutes after sunrise Clear Day (no clouds) Boundary Layer Cumulus Preliminary Findings: We appear to have identified boundary layer “roll” turbulent features that produce  e variations at this AERI site at periodic intervals. Ongoing work will evaluate the horizontal scales of these roll structures based on bulk stability parameters (e.g., Ri). Time Difference (min) Correlation

25. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin “GVision” – A Tool for Coalescing GIFTS-IOMI Data Purpose: – Draw together disparate data for viewing atmospheric parameters – Capitalize on in-house expertise for visualizing GIFTS-IOMI data – Test and validate all models using within UW-MURI (NWP, RTE, etc.) – Develop 3rd-order fields (i.e. slantwise visibility) “VisAD” Capabilities: Java-based application, developed at UW, that is designed for the optimal manipulation and display of large meteorological data sets.

26. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin Goal for Indian Ocean METOC Imager Point Weather Data Marine Inversion Aerosol & dust detection Flight-level & directional visibility Flight-level turbulence SST for engine efficiency Surface characterization IOMI-GIFTS 4 km “Cube”

27. Workshop on Hyperspectal Meteorological Science of UW-MURI and Beyond 14–15 May 2002, Madison, Wisconsin We need to be thinking “Big” –NWP to support IOMI-GIFTS for large domains –IOMI-GIFTS data flow within an Modeling system NWP for next generation IOMI –IOMI-GIFTS validation experiments: NWP support –NWP to develop “turn-key” fleet-ready data system First Progress of Atmospheric Parameters –T, q, winds, stability, turbulence & –Continue from here... In Conclusion...