Robert Spurr, RT Solutions Inc. 9 Channing Street, Cambridge, Mass. Update on Status of (V)LIDORT (and other) RT Models, including enhanced PCA-RTM Robert Spurr, RT Solutions Inc. 9 Channing Street, Cambridge, Mass. Third TEMPO Science Team Meeting RAPCD, UAH, 27-28 May 2015

Quick LIDORT Family Overview (2 slides) Outline of Talk Quick LIDORT Family Overview (2 slides) LIDORT/VLIDORT main codes Other codes (RRS, FO,2STREAM,2OS,Mie/Tmatrix) Upgrades to codes (3 slides) New releases for RT models Thread-safe, OpenMP usage in all codes Upgrades to BRDF and SLEAVE supplements Accelerated RT using PCA – new work (4 slides) Accelerated RT using PCA – new developments 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

LIDORT Family Overview (1): LIDORT/VLIDORT LIDORT and VLIDORT are multiple-scattering linearized discrete ordinate radiative transfer codes in stratified atmospheres. LIDORT is scalar (no polarization); generates radiance I. VLIDORT is vector (with polarization); generates Stokes 4-vector [I,Q,U,V] The pseudo-spherical approximation - solar attenuation (before scattering) treated spherically. Diffuse (multiple) scattering in plane-parallel medium. Precise single scattering – Nakajima/Tanaka ansatz, delta-M scaling. “Linearized codes”  =simultaneously generate any number of analytically derived Jacobians (partial derivatives of I,Q,U,V) w.r.t. profile quantities (e.g. O3) or total-atmosphere “column” quantities (e.g. total O3), or surface properties (e.g. albedo). Atmospheric and surface thermal emission, with Planck functions specified at surface and atmospheric levels. New in 2.7/3.7 release: profile Jacobians w.r.t. Planck functions in multiple-scattering situations  analytic temperature Jacobians. The LIDORT/VLIDORT BRDF and “surface-leaving” supplements are separate modules providing necessary inputs to the main RT codes code. New in 2.7/3.7 release: white-sky and black-sky albedo scaling, revised water-leaving code. Also, mean-value output is available (regular/actinic fluxes+ Jacobians thereof). Currently: Versions 3.7 (LIDORT), 2.7 (VLIDORT). Internet: www.rtslidort.com. Codes in public domain. Available from RT SOLUTIONS (rtsolutions@verizon.net) 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

LIDORT Family Overview (2): Other codes LIDORT-RRS code is a scalar multiple-scattering RT code with rotational-Raman scattering by air molecules. Photons scattered inelastically just once. Compatible with LIDORT when RRS is turned off. Linearized in 2011; This is research code. FO (First-order) model: stand-alone fast/accurate fully linearized single scattering codes. Thermal Emission, BRDFs, etc. Fully compatible with other models, now featured as an exact single-scatter option inside of VLIDORT Version 2.7. 2STREAM is a fast multiple-scatter-only scalar code for radiances and fluxesl. Just 2 discrete ordinates(up/down), fully linearized, pseudo-spherical, thermal emission, BRDFs and surface-leaving supplements, OpenMP compatible. 2OS is a second-order-of-scattering code, developed originally for OCO in 2007 by V. Natraj and R. Spurr. Fast generation of intensity corrections and Q/U estimates. First release as a supplement to VLIDORT in 2015 – fully linearized, OpenMP. Linearized Mie and T-matrix codes . Monochromatic or polydisperse. Various PSDs (lognormal, etc.). Full Bimodal. Jacobians of optical properties w.r.t. microphysical aerosol quantities: refractive index, shape factor (T-matrix), and PSD parameters. 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

Upgrades (1): New releases, RT Models New versions 3.7 (LIDORT) and 2.7 (VLIDORT) released concurrently in early 2015, with similar upgrades for both : Taylor-series expansions for closely-adjacent polar-stream directions; purpose - to avoid spikes and singularities (especially in Jacobians). Expansions now extended and generalized to all cases. New facility for generating Black-body (Planck function) Jacobians in the thermal regime including multiple scattering. Functional in LIDORT, still requires testing VLIDORT. Supplement upgrades: White-sky and black-sky albedos, albedo-scaling options, additional kernels (BRDF); rough-surface extensions to the water-leaving source terms (SLEAVE). Thread-safe code- usable in OpenMP parallel-computing environments (gfortran and ifort, OpenMp Version > 3.1) (VLIDORT only.) Now includes stand-alone FO model (1.4) as alternative SS option. 2STREAM (2.4), FO (1.4) and (new!) 2OS (1.0) codes have all had version upgrades. All codes are OpenMP thread-safe. 2OS Upgrade 2015 (Version 1.0)  validation completed, OpenMP, first release. 2STREAM Upgrade 2015 (Version 2.4)  surface-leaving, new BRDF, OpenMP FO Upgrade 2015 (Version 1.4)  multiple geometry, part of VLIDORT 2.7 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

Upgrades (2): Running in OpenMP Need for OpenMP distributed parallel-computing environment is driven by User feedback for enhanced performance of LIDORT and VLIDORT in hyperspectral applications requiring many RT simulations. Typically, only optical inputs change from wavelength to wavelength  thread setups in OpenMP take up little overhead  good scalability. Upgrades 2015  LIDORT Version 3.7 was made thread safe in June 2014 before current release, VLIDORT 2.7 by December 2014. Upgrades 2015 FO, 2STREAM and 2OS have all been made OpenMP compatible with thread-safe code. Work done early 2015. In package releases, all 5 RTMs contain examples of OpenMP usage, with dedicated drivers having a wavelength loop placed in an “OpenMP Box”. Coming soon, thread-safe code for the linearized Mie and T-matrix codes (especially the latter, which is quite slow). See also the PCA-RTM (next) LIDORT. J.Lin et al. (2014/2015) POMINO algorithm, DLR (W. Zimmer) UPAS ground-processing system (operational), PCA applications with JPL (V. Natraj) VLIDORT. DLR (S. Garcia) ROCINN O2 A-band cloud-scene reflectances, CAS-IAP (Z. Cai) TANSAT algorithm, Tools for the GEMS projects (J. Kim et al.) 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

Upgrades (3): BRDF/SLEAVE supplements BRDF supplement generates the total BRDFs required for running LIDORT/VLIDORT. Exact BRDF r(m,m0,f-f0) for direct-bounce term, BRDF Fourier components rm(mi,mj) for the reflection of diffusely-scattered light. BRDF supplement is fully linearized with respect to surface properties (e.g. wind speed) BRDF code formerly inside the RT models; now separate. Better modularity, easy to change. BRDFs based on 3-kernel MODIS-type inputs (e.g. Ross-thin, Li-sparse), with additional options for glint reflectance. Kernels are semi-empirical models, to be used with caution (e.g. negative values). Upgrades for 2015: (1) Facility for scaling the calculated 3-kernel total BRDFs with either white-sky or black-sky albedo is now in all MS models. (2) BPDF (land surface) options extended to all MS models. SLEAVE (Surface-leaving) supplement generates (sun-normalized) radiance source at the surface, for direct and diffuse light. E.g: fluorescence from vegetation; water-leaving light. Also fully linearized. Upgrades for 2015: now installed in both new versions of LIDORT/VLIDORT - rough surface treatment including whitecap correction, more recent OO modeling and finite transmittance across the surface. BRDF and SLEAVE supplements now present in 2STREAM and FO models, completely consistent with usage in LIDORT and VLIDORT. 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

Accelerated RT using PCA (Example) Set of profile optical properties (e.g., optical depths + single scattering albedos, 325-335 nm Rayleigh + O3 absorption). Remove mean of this set, then perform PCA on it Use small set of “dominant” profiles (Mean and first few significant EOFs) as input to LIDORT and 2Stream multiple-scatter calculations  Use Principal Components to get correction factors  Use 2Stream and FO to get approximate fast-model radiances, then correct. Example: Relative differences in retrieved total ozone for one GOME-2 orbit; PCA using 1, 2, 3, 4 EOFs. Performance enhancements 4.16, 3.83, 3.50 and 3.08 compared to “full”. 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

Accelerated RT using PCA (AMF Example) Example (data from X. Liu) 8001 points 270-350 nm, 11 bins for PCA, determined at decadal intervals of Log(total optical depth). Albedo 10%, Rayleigh atmosphere + ozone (340 DU). 8 half-space streams, 47 layers, 2 geometries. VLIDORT vector mode, 2S in scalar mode  only correction factors for I. Looking at AMFs Timing FO VLIDORT extras 2S PCA TOTAL Faster Exact 0.50 4799.23 0.10 --- --- 4799.88 --- 1 Eof 0.43 19.71 0.03 3.57 1.53 25.29 x 189.8 2 Eof 0.42 32.86 0.03 3.57 1.52 38.43 x 124.9 3 Eof 0.46 46.11 0.03 3.50 1.53 51.57 x 92.9 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

New: OpenMP for Accelerated RTM PCA (1) Recently, the PCA tools developed with NASA JPL have been extended to cover the entire UV-Vis-SWIR spectrum, and the thermal–IR spectrum. Separate tools for radiances and fluxes; 4 tools in all. All 4 tools have required re-engineering for thread-safe OpenMP usage  involves re-structuring major do-loops (over wavelengths and PCA-bins) to set up regions of parallel computing (“OpenMP Boxes”) in the code. After the PCA has been done on optical properties, then, for all bins and all EOFs, the set of all LIDORT/2STREAM calculations based on PCA-derived “super profiles” has been placed in the first OpenMP box. One box only. After the PCA correction factors are established, the fast all-wavelength 2STREAM calculations (and for radiances, the SINGLE-SCATTER calculations) are placed in a second series of OpenMP Boxes (one box for each bin). Some PCA operations have not been not parallelized (yet), e.g. PCA step for “super-profiles”. Care needs to be taken over defining the content and size of the OpenMP shared memory for each OpenMP box, and to make sure that the StackSize is big enough. 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

New: OpenMP for Accelerated RTM PCA (2) 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015

New: OpenMP for Accelerated RTM PCA (3) Whole 4-tool system recently completed last week, currently undergoing shakedowns at JPL and RT Solutions. Same approach for PCA Intensity applications can be applied to fast calculation of all manner of Jacobians. Before an extended-wavelength simulation using a PCA tool, one needs to create large amounts of hyperspectral optical properties, for example based on data sets compiled for the Geocape project. This “Create-Properties” operation is time-consuming in its own right, starting to apply OpenMP optimization to it. The HITRAN cross-section routine is the most time-critical step. Here, the Do-loop over height levels (Pressure/Temperature) has been parallelized. Typical performance ~3x faster for 4 cores. 9/19/2018 R. Spurr. RT Model Upgrades. Third TEMPO STM, RAPCD, UAH, 27-28 May 2015