1. Bill Campbell and Liz Satterfield 13 th JCSDA Technical Review Meeting and Science Workshop on Satellite Data Assimilation 15 May 2015 Accounting for Correlated Satellite Observation Error in NAVGEM 1

2. 2 Sources of Observation Error IMPERFECT OBSERVATIONS True Temperature in Model Space T=28°T=38°T=58° T=30°T=44°T=61° T=32°T=53°T=63° T=44° 1)Instrument error (usually, but not always, uncorrelated) H 2)Mapping operator (H) error (interpolation, radiative transfer) 3)Pre-processing, quality control, and bias correction errors 4)Error of representation (sampling or scaling error), which can lead to correlated error:

3. 3 Correlated Error in Operational DA R Until recently, most operation DA systems assumed no correlations between observations at different levels or locations (i.e., a diagonal R) To compensate for observation errors that are actually correlated, one or more of the following is typically done: – Discard (“thin”) observations until the remaining ones are uncorrelated (Bergman and Bonner (1976), Liu and Rabier (2003)) – Local averaging (“superobbing”) (Berger and Forsythe (2004)) – Inflate the observation error variances (Stewart et al. (2008, 2013) R Theoretical studies (e.g. Stewart et al., 2009) indicate that including even approximate correlation structures outperforms diagonal R with variance inflation * In January, 2013, the Met Office went operational with a vertical observation error covariance submatrix for the IASI instrument, which showed forecast benefit in seasonal testing in both hemispheres (Weston et al. (2014))

4. 4 R From O-F, O-A, and A-F statistics from any model (e.g. NAVGEM), the observation error covariance matrix R, the representer HBH T, and their sum can be diagnosed R This method is sensitive to the R and HBH T that is prescribed in the DA system An iterative approach may be necessary R 1 R Diagnose R 1, which will be different from the original R R 1 Symmetrize R 1, possibly adjusting its eigenvalue spectrum R 1 Implement R 1 and run NAVGEM R 2 R true Diagnose R 2, which we hope will be closer to R true Desroziers Method (Desroziers et al. 2005)

5. 4DVar Primal Formulation Scale by B -1/2 4D-var iteration is on this problem -- We need to invert R!

6. 4DVar Dual Formulation Scale by R -1/2 C 4D-Var iteration is on this problem – No need to invert C! Iteration is done on partial step and then mapped back with BH T An advantage of the dual formulation is that correlated observation error can be implemented directly No matrix inverse is required, which lifts some restrictions on the feasible size of a non-diagonal R In particular, implementing horizontally correlated observation error is significantly less challenging

7. 7 Correlated Observation Error and the ATMS Advanced Technology Microwave Sounder (ATMS) 13 temperature channels 9 moisture channels

8. Current observation error correlation matrix used for ATMS, and for ALL observations 8 Error Covariance Estimation for the ATMS Statistical Estimate Channel Number 4 5 6 7 8 9 10 11 12 13 14 15 18 19 20 21 22 Channel Number 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 4 5 6 7 8 9 10 11 12 13 14 15 18 19 20 21 22 Desroziers’ method estimate of interchannel portion of observation error correlation matrix for ATMS 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 4 5 6 7 8 9 10 11 12 13 14 15 18 19 20 21 22 Channel Number Current Treatment Channel Number Temperature Moisture

9. Current observation error correlation matrix used for ATMS, and for ALL observations 9 Iterating Desroziers Old Statistical Estimate Channel Number 4 5 6 7 8 9 10 11 12 13 14 15 18 19 20 21 22 Channel Number 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 4 5 6 7 8 9 10 11 12 13 14 15 18 19 20 21 22 Desroziers’ method estimate of interchannel portion of observation error correlation matrix for ATMS 4 5 6 7 8 9 10 11 12 13 14 15 18 19 20 21 22 Channel Number New Statistical Estimate Channel Number The change is not large, which is (weak) evidence that the procedure may converge

10. Convergence and the Cauchy Interlacing Theorem Cauchy interlacing theorem Let A be a symmetric n × n matrix. The m × m matrix B, where m ≤ n, is called a compression of A if there exists an orthogonal projection P onto a subspace of dimension m such that P*AP = B. The Cauchy interlacing theorem states:compression Theorem. If the eigenvalues of A are α 1 ≤... ≤ α n, and those of B are β 1 ≤... ≤ β j ≤... ≤ β m, then for all j < m + 1, Notice that, when n − m = 1, we have α j ≤ β j ≤ α j+1, hence the name interlacing theorem. Cauchy interlacing theorem Let A be a symmetric n × n matrix. The m × m matrix B, where m ≤ n, is called a compression of A if there exists an orthogonal projection P onto a subspace of dimension m such that P*AP = B. The Cauchy interlacing theorem states:compression Theorem. If the eigenvalues of A are α 1 ≤... ≤ α n, and those of B are β 1 ≤... ≤ β j ≤... ≤ β m, then for all j < m + 1, Notice that, when n − m = 1, we have α j ≤ β j ≤ α j+1, hence the name interlacing theorem. What happens when radiance profiles are incomplete (i.e., at a given location, some channels are missing, usually due to failing QC checks)?

11. Condition Number Constrained Correlation Matrix Approximation We want to find a positive definite approximation to the matrix is minimized. In the trace norm, simply set all of the smallest singular values equal to the value that gives the desired condition number, and then reconstruct the matrix with the singular vectors to obtain the approximate matrix. Another method, used by Weston et al. 2014, is to increase the diagonal values of the matrix until the desired condition number is reached. We believe there is better theoretical justification for the first method using the Ky Fan p-k norm (Tanaka, M. and K. Nakata, 2014: “Positive definite matrix approximation with condition number constraint”, Optim. Lett. 8, pp 939-947) The Ky Fan p-k norm of where denotes the i th largest singular value of When p=2 and k=n, it is called the Frobenius norm; when p=1 and k=n, it is called the trace norm.

12. 12 Conjugate Gradient Convergence Goal

13. 13 We ran NAVGEM 1.3 at T425L60 resolution with the full suite of operational instruments for two months, from July 1, 2013 through Aug 31, 2013 R The control experiment (atid) used a diagonal R for the ATMS instrument R The initial ATMS experiment (atms) used the R diagnosed from the Desroziers method applied to three months of innovation statistics R The second ATMS experiment (atms2) used the R diagnosed from the Desroziers method applied to the first experiment Results from these experiments were neutral, but… Experimental Design

14. 14 Current Results with Proposed Scorecard

15. 15 Using the Desroziers Diagnostic for IASI Channel Selection Water vapor channels 2889, 2994, 2948, 2951, and 2958 have very high error correlation (>0.98) The eigenvectors corresponding to the 4 smallest eigenvalues project only on to these 5 channels It makes sense to use the Desroziers diagnostic to do a posteriori channel selection, which has the bonus of improving the condition number of the correlation matrix, and thus solver convergence

16. 16 Conjugate Gradient Convergence Goal

17. 17 We chose to assimilate only Channel 2889 (iasid_oneh20). This resulted in faster convergence of the 4DVar solver, as well as +1 on the FNMOC scorecard. R The initial IASI experiment (iasicor_oneh20) used the R diagnosed from the Desroziers method applied to three months of innovation statistics, and yielded +2 on the FNMOC scorecard. Experimental Design

18. 18 Current Results with Standard Scorecard

19. 19 The Desroziers error covariance estimation methods can quantify correlated observation error We can make minimal changes to diagnosed error correlations to fit operational time constraints R The NAVGEM system allows for direct use of a non-diagonal R; implementing vertically correlated error is straightforward Correctly accounting for correlated observation error in data assimilation may yield superior forecast results without a large computational costConclusions

20. 20 Summary of Proposed Work Estimate IASI and ATMS vertical observation error covariance Estimate IASI and ATMS vertical observation error covariance Implement IASI and ATMS vertical observation error covariance in NAVDAS-AR Implement IASI and ATMS vertical observation error covariance in NAVDAS-AR Testing, debugging, exploring solver convergence properties, etc. Testing, debugging, exploring solver convergence properties, etc. Evaluate preliminary results with observation sensitivity tools and standard forecast metrics Evaluate preliminary results with observation sensitivity tools and standard forecast metrics

21. 21 Hollingsworth-Lönnberg Method (Hollingsworth and Lönnberg, 1986) Use innovation statistics from a dense observing network Assume horizontally uncorrelated observation errors Calculate a histogram of background innovation covariances binned by horizontal separation Fit an isotropic correlation model, extrapolate to zero separation to estimate the correlated (forecast) and uncorrelated (observation) error partition Extrapolate red curve to zero separation, and compare with innovation variance (purple dot) Mean of ob minus forecast (O-F) covariances, binned by separation distance u2u2 c2c2 Assumes no spatially-correlated observation error

22. 22 analyses forecasts Improvements to global analyses and forecasts from better use of data with correlated error NAVGEM Transition of a new component of our global system (NAVGEM) to account for vertical error correlation Anticipated Results

23. 23 Conjugate Gradient Convergence Control Experiment 20130802002013080200 Log

24. 24 Conjugate Gradient Convergence 2013 7/1 – 8/30

25. 25 Conjugate Gradient Convergence

26. 26 Conjugate Gradient Convergence

27. Assess Forecast Impact with Observation Sensitivity Tools Adjoint-based system (Langland and Baker, 2004) enables rapid assessment of changes to the DA system Jul01-Aug02, 2013 Add vertical correlated error to ATMS channels Add vertical correlated error to ATMS channels NAVDAS-AR Results

28. 28 How can we best estimate errors in Desroziers/Hollingsworth- Lönnberg diagnostics? – Should we expect agreement between different methods? – Will the Desroziers diagnostic converge if both R and B are incorrectly specified? – Amount of data required to estimate covariances? Seasonal dependence? – Best methods to symmetrize the Desroziers matrix? How to gauge improvement? – Do we also need to adjust to see overall improvement to the system? – How do we maintain the correct ratio for DA? What about convergence? – Should we do an eigenvalue scaling to improve the condition number? Discussion

29. 29 Implement R and Test We have already implemented both the Desroziers and H-L covariance estimation methods, although some modification of H-L will be needed for vertical error covariances We have identified the portion of the NAVDAS-AR code that must be changed in order to implement vertical observation error covariances Because of our formulation, no matrix inverses are required The capability to assimilate ATMS and IASI with vertically correlated error will jump start the research program proposed in the Correlated Observation Error 6.2 New Start Other weather centers NRL Other weather centers NRL

30. 30 R From O-F, O-A, and A-F statistics, the observation error covariance matrix R, the representer HBH T, and their sum can be diagnosed R This method is sensitive to the R and HBH T that is prescribed in the DA system An iterative approach may be necessary Desroziers Method (Desroziers et al. 2005) R R