1. 1 Progress on the Optimal Spectral Sampling (OSS) Method Jean-Luc Moncet AER, Inc.

2. Atmospheric and Environmental Research, Inc. 2 Topics Introduction OSS S/W Transition to JCSDA Generalized training Clear/cloudy training Inversion issue Treatment of multiple scattering Validation against CHARTS Application to AIRS Trace gases Summary/future work

3. Atmospheric and Environmental Research, Inc. 3 Overview of the OSS approach OSS method (Moncet et al. 2003, 2001) models the channel radiance as Wavenumber n i (nodes) and weights w i are determined by fitting “exact” calculations (from line-by-line model) for globally representative set of atmospheres (training set) Radiance training is fast and provides mechanism for directly including slowly varying functions (e.g. Planck, surface emissivity) in the selection process

4. Atmospheric and Environmental Research, Inc. 4 Relationship between OSS and ESFT/correlated-k methods Extension to multiple absorbers along inhomogeneous path (e.g. Armbruster and Fisher, 1996) ESFT (Wiscombe and Evans, 1977) for single layer, single absorber case: OSS solution: Extension of ESFT to inhomogeneous atmospheres with multiple absorbers reduces the problem to a single (wavenumber) dimension and ensures that the solution is physical (i,k) = (2,1)(2,2) i = 2 i = 1 i = 3 (2,3)(2,4) (2,5) Selected nodes

5. Atmospheric and Environmental Research, Inc. 5 Optimal Spectral Sampling (OSS) method OSS absorption parameterization leads to fast and numerically accurate RT modeling: OSS-based RT model can approach line-by-line calculations arbitrarily closely Adjustable numerical accuracy: Possibility of trade off between accuracy and speed Unsupervised training No empirical adjustment: cuts significantly on cost of testing approximations and validating model Provides flexible handling of (variable) trace molecular species Designed to handle large number of variable trace species w/o any change to model – low impact on computational cost Selection of variable trace gases at run time Memory requirements do not change whether we are dealing with one or more instruments Execution speed primarily driven by total spectral coverage and maximum spectral resolution (not by number of instruments) Leads to accurate handling of multiple scattering (cloudy radiance assimilation)

6. 6 OSS S/W Transition to JCSDA

7. Atmospheric and Environmental Research, Inc. 7 OSS delivery status OPTRAN/OSS intercomparison AIRS, GOES, HIRS, AMSU, SSMIS Results presented last year indicated significant speed/accuracy advantage of OSS Larger memory requirements (single instrument) Supported transition of OSS molecular absorption parameterization into CRTM Dynamic array allocation implemented Revised/adapted interface parameters Develop pre-processing S/W to merge multiple instruments into single file and remove redundant nodes and reformat OSS coefficient files OSS files delivered for AMSU, GOES, HIRS, SSMIS and AIRS First CRTM/OSS version integrated at NOAA

8. Atmospheric and Environmental Research, Inc. 8 OSS (IR) Parameter Generation S/W diagram

9. Atmospheric and Environmental Research, Inc. 9 OSS Parameter Generation Delivery Status Revised/adapted and delivered OSS generation S/W Microwave (full code) with Rosenkranz spectroscopy Infrared: Part 2 – localized training Part 1 (partial delivery) Other items supplied: LBLRTM radiances and optical depth files for 0.01 cm -1 boxcars covering the spectral range encompassing current operational instruments channels for use with Selection Part 2 Delivered generation S/W successfully ran at NOAA

10. 10 Generalized Training for High Spectral Resolution IR Sounders

11. Atmospheric and Environmental Research, Inc. 11 Generalized training Goal: Speed up RT calculations/radiance inversion for high resolution IR sounders (e.g. AIRS, IASI, CrIS) Case considered: Complete channel set (no information loss)

12. Atmospheric and Environmental Research, Inc. 12 Localized versus non-localized training Localized training (current) operates on individual channels, one at a time – node redundancy due to overlapping ILS AIRS (2378 channels): Average # nodes per channel: ~9 nodes/channel Total # of nodes/# of channel (i.e. no redundancy) = 1.9 Non-localized training operates on groups of N channels (up to full channel set) Exploits node-to-node correlation to minimize total number of nodes across a spectral domain (regression!!!) Results in significant increase in number of points in any given channel increases Speed gain varies with surface complexity and differ for clear (or cloud cleared) and cloudy radiances Gain higher for ocean/highly emissive land surfaces (~flat emissivity spectrum) Minimum gain ~2 for AIRS when considering no correlation on scales larger than 20 cm -1 Nominal accuracy = 0.05K

13. Atmospheric and Environmental Research, Inc. 13 Performance example (AIRS) Localized training (0.05K accuracy): ~2nodes /channel ~5000 monochromatic calculations for full AIRS channel set Generalized training: ~0.1 node/channel Reduces number of monochromatic calculations to ~250 Speed gain in RT computation ~ 20 compared to localized training for AIRS (ocean/clear)

14. Atmospheric and Environmental Research, Inc. 14 Examples of error spatial distribution

15. Atmospheric and Environmental Research, Inc. 15 Non-localized cloudy training Must include slowly varying cloud/aerosol optical properties in training Over wide bands: training can be done by using a database of cloud/aerosol optical properties More general training obtained by breaking spectrum in intervals of the order of 10 cm -1 in width (impact of variations in cloud/aerosol properties on radiances is quasi-linear) and by performing independent training for each interval lower computational gain but increased robustness Direct cloudy radiance training not recommended ! Clouds tend to mask molecular structure which makes training easier If “recipe” for mixture of clear and cloudy atmospheres in direct training not right: clear-sky performance degrades

16. Atmospheric and Environmental Research, Inc. 16 Robust, physical approach for including slowly varying functions (e.g. cloud optical properties, surface emissivity) into OSS formalism Cloudy training preserves clear-sky solution k = 1 2 3 4 5 Alternate two-step training preserves clear-sky solution First step: normal clear-sky (transmittance/radiance) training to model molecular absorption Second step: duplicate + redistribute nodes across spectral domain and recompute weights to incorporate slowly varying functions into the model Single/multi-channel cloudy training over wide spectral domains

17. Atmospheric and Environmental Research, Inc. 17 Alternatives (application dependent): A.PC: Generalized training operates on channel radiances or PC’s Use of PC’s reduces first dimension of A-matrix Further speed gain in inversion B.Operate directly in node space More robust No risk of filtering out important spectral features Channels are inspected individually after training and local nodes are added where needed to improve performance Avoids Jacobians transformation all together and reduce K-matrix size (inversion speed up) for AIRS: 2378 channels -> 250 nodes Inversion Variational retrieval methods: Average channel uses ~150 nodes (instead of ~9 for localized training) Mapping Jacobians from node to channel space partially offsets speed gain

18. Atmospheric and Environmental Research, Inc. 18 Clear/ocean test Example of retrieval performance (AIRS) – constant noise Channel space retrieval Node space retrieval Little impact on temperature and water vapor retrieval performance Overall speed gain (RT model + inversion) ~15 Need strategy for handling input dependent noise (application dependent) SW band noise most sensitive to scene temperature Crude classification (or no classification) may be sufficient in the clear sky Cloud clearing noise amplification (?)

19. 19 Application to Scattering Atmospheres

20. Atmospheric and Environmental Research, Inc. 20 CHARTS (Moncet and Clough, 1997): Fast adding-doubling scheme for use with LBLRTM Uses tables of layer reflection/transmittance as a function of total absorption computed at run time Validation against measurements from Rotating Shadowband Spectroradiometer (RSS) spectra at the ARM/SGP site OSS/CHARTS Comparison

21. Atmospheric and Environmental Research, Inc. 21 OSSSCAT: Single wavelength version of CHARTS (no spectral interpolation) Cloudy validation: Molecular absorption from 740-900 cm -1 domain 1cm -1 boxcars, thermal only Cloud extinction OD range: 0-100 Example: 780-860 cm -1 Low cloud case (925- 825 mb) OSS/CHARTS Comparison (2)

22. Atmospheric and Environmental Research, Inc. 22 Same as previous High cloud case (300-200 mb) Clear sky training adequate in thermal regime Refinement in training needed for thick clouds (OD > 50) when SSA approaches 1 and high scan angles OSS/CHARTS Comparison (3)

23. Atmospheric and Environmental Research, Inc. 23 MADA** optical properties (tropical cirrus) **References: (Mitchell, 2000, 2002)

24. Atmospheric and Environmental Research, Inc. 24 Application to AIRS Single FOV 1DVAR retrieval Atmosphere/SST from NCEP/GDAS Adjusted parameters: Cloud top/thickness Ice particles effective diameter (Deff) IWP Effective temperature MODIS 1 st guess AER/SERCAA cloud algorithms RTM: OSSSCAT (100 layers) 4-streams GOES imagery

25. Atmospheric and Environmental Research, Inc. 25 MODIS retrieved cloud product

26. Atmospheric and Environmental Research, Inc. 26 Calculated vs. measured cloudy AIRS spectra AIRS (896 cm -1 ) brightness temperature

27. 27 Trace Gases

28. Atmospheric and Environmental Research, Inc. 28 Trace gases RT model designed to handle any number of variable trace species Adding a new variable species requires no change in OSS parameterization No change in RTM required Only need to include variability in training (number of nodes may increase as a result) # of variable trace gases and molecule type specified on node-by-node basis (set by the user at run time) Average number of trace gases absorbing at any given frequency << total number of absorbing species in the atmosphere Computationally efficient and minimizes memory requirements Inexpensive Jacobian computation:

29. Atmospheric and Environmental Research, Inc. 29 Example of trace gas profiles CO 2, CH 4, N 2 O and CO profiles from combined troposphere/stratosphere model from NASA Global Modeling Initiative (GMI)** Geographically/seasonally matched to existing temperature water vapor training set ** Data provided by Susan Strahan, Jose Rodriguez, and Bryan Duncan

30. Atmospheric and Environmental Research, Inc. 30 Future Projections Account for the secular increase in CO 2, CH 4, and N 2 O, in training (avoids having to periodically retrain the model) by running model into future years Alternative approach: scale the current profiles by a constant factor equal to take into account projected increase in surface concentration May require height-dependent scaling based on the age of air (small effect) CO projections are highly model- dependent (since CO is more affected by chemistry) and harder to quantify (e.g., biomass burning). Current variability is probably representative of the near future. Projections of future surface concentrations from IPCC (2001)

31. Atmospheric and Environmental Research, Inc. 31 Summary/future work Top priorities: Continue supporting OSS integration in CRTM framework Provide support for efficiency improvement Handling of “slow functions” Handling of variable input pressure grid Adapt RT model structure Complete coefficient generation (localized training) S/W delivery and provide required support Generalized (non-localized) training Provided brackets for potential gain in speed/memory requirements Gain depends of surface type and presence of clouds/aerosols Ongoing testing in AER 1D-VAR system with AIRS Started testing with NPOESS/CrIS Cloudy conditions Completing preliminary validation of OSS in cloudy environment (infrared) Applied to AIRS cloud property retrieval Improve speed (timing dominated by RT solution): Generalized training Fast parameterization of scattering effects (already available for EO imaging instruments) Started validation effort in microwave (NPOESS/CMIS)