Physical retrieval (for MODIS) Andy Harris Jonathan Mittaz Prabhat Koner (Chris Merchant, Pierre LeBorgne)

Satellite data – pros and cons Main advantages of satellite data –Frequent and regular global coverage (cloud cover permitting for IR) –Not a direct measure. A retrieval process is required –‘Single’ source of data –Many observations Challenges –Single source + many observations means that data must be accurate, or risk swamping the conventional record with erroneous values –Lack of other sources in remote regions to cross- compare

Reprocessing of historical data Unless we have one of these… …we must reprocess old data to the standard required for climate monitoring Expected SST trend is ~0.2 K/decade Hence requirement is that observing system must be stable to <0.1 K/decade AVHRR (A)ATSR MODIS El ChichonMt PinatuboEl Niño

The chief advantage of radiative transfer (RT) is that it allows specification of the retrieval algorithm without bias towards the data-rich regions The in situ data can then act as a random independent sampling of the retrieval conditions If the observed errors agree with the modeled ones, then high confidence can be placed on the modeled errors in data-sparse regions Radiative transfer-based retrievals Is it necessary? Let’s consider the likely errors in the empirical ‘state-of-the-art’ AVHRR Pathfinder SST…

Early theory required SST – T i = k i F(atm) This allowed SST = k 2 T 1 – k 1 T 2 ——————————— (k 2 – k 1 ) And hence the “split- window” equation, mystique about channel differences, etc. Only need to assume SST – T i  SST – T j to get SST = a 0 +  a i T i Some refinements to account for non-linearity, scan angle: SST = a 0 + a 1 T 11 + SST bg a 2 (T 11 – T 12 ) + ( SZA-1)a 3 (T 11 – T 12 )

Some issues w.r.t. regression-based SST retrievals Does it matter what form the scan-angle correction takes? – Sec(ZA) term multiplying a channel difference (e.g. NLSST) – Separate SZA-dependent term for each independent variable How much signal is coming from the surface? – Dependence on water vapor – Geophysical coupling

Scan-angle dependence Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 In-sample testing of algorithm accuracy

Scan-angle dependence Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 In-sample testing of algorithm accuracy

How much signal? Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 The mid-IR ~4 μm channel(s) are powerful…

How much signal? Simulation using Modtran & 1,358 globally sampled ECMWF profiles, sec(ZA) = 1.0, 1.25, 1.5… …2.5 The mid-IR ~4 μm channel(s) are powerful… Use all surface-sensitive channels…

Simulated Pathfinder Retrieval Errors The effect can be modeled using the following –Pathfinder matchup dataset –Full-resolution ERA-40 atmospheric and surface data –Fast-forward radiative transfer model (CRTM) Process is as follows –Pathfinder matchup information is used to select the appropriate atmospheric profiles for each month –SSTs and profiles are used to generate ToA radiances which are then used to generate algorithm coefficients –Retrieval coefficients are then applied to full month of radiances generated from cloud-free ERA-40 data AVHRR Pathfinder Oceans is current “state-of-the-art” SST –Distribution & quantity of matchup data change over time –How does this affect SST retrieval performance?

Pathfinder Mean Bias 1985 – 1999

Changes in sampling 1985 – 1999 Physically-based methodology models retrieval effects explicitly – apply to whole AVHRR record to improve consistency of SST record for climate studies

Using Radiative Transfer in SST Retrieval Since algorithm biases can be modeled, just subtract them – Has been employed operationally (LeBorgne et al., 2011) – Has little impact on S.D., no impact on sensitivity, but does improve bias – Dependent on model input (!) Accuracy of NWP profile Not in model state  not corrected (e.g. aerosol)

Direct removal of local retrieval bias Example - Methodology of Meteo-France: [From presentation by Pierre XIII, Tokyo]

“Physical” Retrieval: Estimate adjustment to “guess” SST Based on local linearization, i.e. Δy = KΔx – Δy = y o – y g, K, y g from RTM + NWP – x is reduced state vector, at least [SST, TCWV] T – K is Jacobian of partial derivatives of y g w.r.t. x Ordinary unweighted least squares solution: Δx = (K T K) -1 K T Δy [ = GΔy ] Potential issues: – G may be ill-conditioned (noise amplification) – K may be “incorrect” – Δy may be biased, i.e. Δy ≠ 0 when Δx = 0 e.g. RTM, calibration

Incremental Regression Only use Δy and simply derive a regression coefficient- based operator to retrieve ΔSST (Petrenko et al.) – Do not retrieve a water vapor adjustment – Side-step issues of calibration, errors in RTM calculation, etc. Essentially no control over algorithm sensitivity – Noisy input will suppress coefficients No means of estimating uncertainty – Cf. other physical methods No means of iterating solution – No adjustment to model state (apart from SST) Revision of other processing elements requires recalculation of coefficients

The “Traditional” Approach Optimal Estimation E.g. Merchant et al. for SST Primary issue is dealing with the ill-conditioned nature of gain matrix G – Use prior estimates of uncertainty in RTM, model state and instrument noise – RTM + satellite errors combined into one covariance matrix S e – Error in model state propagated through covariance for reduced state vector S a Δx = (K T S e -1 K + S a -1 ) -1 K T S e -1 Δy Error covariance matrices function as a regularization operator on the gain matrix – Reduces the “condition number” – Any regularization will reduce sensitivity to true SST change

Extended OESST [From presentation by Chris XIV, Woods Hole]

Extended OESST

Extended OESST - Results

MODIS - OSTIA, Feb 2012

[SST (night) – OSTIA] cf. modeled Modeled clear-sky NLSST bias is February average for 1985 – 1999 −No aerosols Several features common to both observed and modeled biases −Boundary currents  over-estimate of gamma −Cold tongue  under-estimate of gamma −High SST but low water vapor −(Decoupling of SST and air temperature) Modeling shows annual cycle of bias

Hence Lat-band coefficients, etc. Employ a physical retrieval methodology −MTLS

Modified Total Least Squares: A Deterministic Regularization Based on local linearization, i.e. G = (K T K + rI) -1 K T – r is dynamically calculated regularization strength r = (2 log(κ)/||Δy||)σ 2 end – κ is the condition number of K – σ 2 end is the lowest singular value of [K Δy] – ||Δy|| is L 2 -norm of Δy Regularization uses observation-based estimate of noise amplification – Closer to “actual” error for retrieval/observation conditions on a case-by-case basis

MTLS applied to MODIS Validation against iQUAM – MTLS: -0.02±0.36 K SST(night): -0.21±0.47 K – Reduction of ~0.3 K of independent error – (SST4: -0.24±0.44 K) – (Difference in bias due to skin effect)

Uncertainty Estimation OE uses “goodness of fit” χ 2 = (KΔx – Δy) T [S e (KS a K T + S e -1 )S e ] -1 (KΔx – Δy) MTLS formulation for total error, e ||e|| = ||(MRM – I)Δx|| + ||G||  ||(Δy - KΔx)||  K T – Δx should be true value, but is substituted by retrieved quantity – MRM is model resolution matrix (a.k.a. averaging kernel)

Summary Inherent limitations in NLSST MODIS has 16 TIR channels – Currently, only a very few used for SST retrieval 1 st cut MTLS physical retrieval shows promise – Initial result subject to MODIS cloud mask Extra channels permit more complex retrieval vector – WV scale height, air temperature, aerosol, etc. Multiple iterations Smoothed inputs (Merchant) OE and MTLS allow prospect for direct uncertainty estimation