1. ECMWF Observation Influence DA School Buenos Aires 2008 slide 1 Influence matrix diagnostic to monitor the assimilation system Carla Cardinali

2. ECMWF Observation Influence DA School Buenos Aires 2008 slide 2 Monitoring Assimilation System ECMWF 4D-Var system handles a large variety of space and surface- based observations. It combines observations and atmospheric state a priori information by using a linearized and non-linear forecast model Effective monitoring of a such complex system with 10 7 degree of freedom and 10 6 observations is a necessity. No just few indicators but a more complex set of measures to answer questions like How much influent are the observations in the analysis? How much influence is given to the a priori information? How much does the estimate depend on one single influential obs?

3. ECMWF Observation Influence DA School Buenos Aires 2008 slide 3 Influence Matrix: Introduction Unusual or influential data points are not necessarily bad observations but they may contain some of most interesting sample information In Ordinary Least-Square the information is quantitatively available in the Influence Matrix Tuckey 63, Hoaglin and Welsch 78, Velleman and Welsch 81 Diagnostic methods are available for monitoring multiple regression analysis to provide protection against distortion by anomalous data

4. ECMWF Observation Influence DA School Buenos Aires 2008 slide 4 Influence Matrix in OLS The OLS regression model is Y (mx1) observation vector X (mxq) predictors matrix, full rank q β (qx1) unknown parameters (mx1) error The fitted response is OLS provide the solution m>q

5. ECMWF Observation Influence DA School Buenos Aires 2008 slide 5 Influence Matrix Properties S (mxm) symmetric, idempotent and positive definite matrix It is seen The diagonal element satisfy Cross-SensitivitySelf-Sensitivity Average Self-Sensitivity=q/m

6. ECMWF Observation Influence DA School Buenos Aires 2008 slide 6 Influence Matrix Related Findings The change in the estimate that occur when the i-th is deleted CV score can be computed by relying on the all data estimate ŷ and S ii

7. ECMWF Observation Influence DA School Buenos Aires 2008 slide 7 Outline Conclusion Observation and background Influence Generalized Least Square method Findings related to data influence and information content Toy model: 2 observations

8. ECMWF Observation Influence DA School Buenos Aires 2008 slide 8 Hx b y HK I-HK Hx b y HK I-HK Solution in the Observation Space The analysis projected at the observation location The estimation ŷ is a weighted mean Hx b y HKI-HK B(q x q)=Var(x b ) R(p x p)=Var(y) K(q x p) gain matrix H(p x q) Jacobian matrix

9. ECMWF Observation Influence DA School Buenos Aires 2008 slide 9 Influence Matrix Observation Influence is complementary to Background Influence

10. ECMWF Observation Influence DA School Buenos Aires 2008 slide 10 Influence Matrix Properties The diagonal element satisfy

11. ECMWF Observation Influence DA School Buenos Aires 2008 slide 11 Synop Surface Pressure Influence >1 S ii >1 due to the numerical approssimation

12. ECMWF Observation Influence DA School Buenos Aires 2008 slide 12 Aircraft 250 hPa U-Comp Influence S ii >1 due to the numerical approssimation >1

13. ECMWF Observation Influence DA School Buenos Aires 2008 slide 13 QuikSCAT U-Comp Influence >1 S ii >1 due to the numerical approssimation

14. ECMWF Observation Influence DA School Buenos Aires 2008 slide 14 Observation Influence: Vertical levels >1 S ii >1 due to the numerical approssimation

15. ECMWF Observation Influence DA School Buenos Aires 2008 slide 15 AMSU-A channel 13 Influence S ii >1 due to the numerical approssimation >1

16. ECMWF Observation Influence DA School Buenos Aires 2008 slide 16 Toy Model: 2 Observations x1x1 x2x2 y2y2 y1y1 Find the expression for S as function of r and the expression of for α=0 and 1 given the assumptions:

17. ECMWF Observation Influence DA School Buenos Aires 2008 slide 17 Toy Model: 2 Observations x1x1 x2x2 y2y2 y1y1 S ii r 1 1 1/2 1/3 1/2 =1 =0 0

18. ECMWF Observation Influence DA School Buenos Aires 2008 slide 18 Consideration (1) Where observations are dense S ii tends to be small and the background sensitivities tend to be large and also the surrounding observations have large influence (off- diagonal term) When observations are sparse S ii and the background sensitivity are determined by their relative accuracies (r) and the surrounding observations have small influence (off-diagonal term)

19. ECMWF Observation Influence DA School Buenos Aires 2008 slide 19 Toy Model: 2 Observations =0 =1 x1x1 x2x2 y2y2 y1y1

20. ECMWF Observation Influence DA School Buenos Aires 2008 slide 20 Consideration (2) When observation and background have similar accuracies (r), the estimate ŷ 1 depends on y 1 and x 1 and an additional term due to the second observation. We see that if R is diagonal the observational contribution is devaluated with respect to the background because a group of correlated background values count more than the single observation (2-α 2 2). Also by increasing background correlation, the nearby observation and background have a larger contribution

21. ECMWF Observation Influence DA School Buenos Aires 2008 slide 21 Global and Partial Influence Global Influence = GI = 100% only Obs Influence 0% only Model Influence Partial Influence = PI = Type Variable Area Level

22. ECMWF Observation Influence DA School Buenos Aires 2008 slide 22 Global and Partial Influence Level 1 1000-850 2 850-700 3 700-500 4 500-400 5 400-300 6 300-200 7 200-100 8 100-70 9 70-50 10 50-30 11 30-0 Type SYNOP AIREP SATOB DRIBU TEMP PILOT AMSUA HIRS SSMI GOES METEOSAT QuikSCAT Variable uvTqpsTbuvTqpsTb Area TropicsS.HemN.Hem EuropeUSN.Atl …

23. ECMWF Observation Influence DA School Buenos Aires 2008 slide 23 Average Influence and Information Content Global Observation Influence GI=15% Global Background Influence I-GI=85% 2003 ECMWF Operational

24. N.Hemisphere PI = 15% Tropics PI = 17.5% S.Hemisphere PI = 12% GI = 15.3% 2003 ECMWF Operational

25. ECMWF Observation Influence DA School Buenos Aires 2008 slide 25 2007 ECMWF Operational 20 16 12 8 4

26. ECMWF Observation Influence DA School Buenos Aires 2008 slide 26 DFS % first bar Global, second North Pole and third South Pole Global Information Content 20% North Pole 5.6% South Pole 1.1%

27. ECMWF Observation Influence DA School Buenos Aires 2008 slide 27 DFS % first bar North Pole and second South Pole Global Information Content 20% North Pole 5.6% South Pole 1.1% Tropics 5.5%

28. ECMWF Observation Influence DA School Buenos Aires 2008 slide 28 Evolution of the B matrix: B computed from EnDA X t + ε Stochastics y+ε o SST+ε SST Xb+εbXb+εb y+ε o SST+ε SST AMSU-A ch6

29. ECMWF Observation Influence DA School Buenos Aires 2008 slide 29 Evolution of the GOS: Interim Reanalysis Aircraft 200-300 hPa 1999 2007

30. ECMWF Observation Influence DA School Buenos Aires 2008 slide 30 Evolution of the GOS: Interim Reanalysis AMSU-A ch6 1999 2007

31. ECMWF Observation Influence DA School Buenos Aires 2008 slide 31 Evolution of the GOS and of the B: Interim Reanalysis AMSU-A ch13 1999 2007 2003

32. ECMWF Observation Influence DA School Buenos Aires 2008 slide 32 Evolution of the GOS: Interim Reanalysis AMSU-A 1999 2007

33. ECMWF Observation Influence DA School Buenos Aires 2008 slide 33 Evolution of the GOS: Interim Reanalysis U-comp Aircraft, Radiosonde, Vertical Profiler, AMV 1999 2007

34. ECMWF Observation Influence DA School Buenos Aires 2008 slide 34 Conclusions The Influence Matrix is well-known in multi-variate linear regression. It is used to identify influential data. Influence patterns are not part of the estimates of the model but rather are part of the conditions under which the model is estimated Disproportionate influence can be due to: incorrect data (quality control) legitimately extreme observations occurrence to which extent the estimate depends on these data S ii =1 Data-sparse Single observation Model under-confident (1-S ii ) S ii =0 Data-dense Model over-confident tuning (1-S ii )

35. ECMWF Observation Influence DA School Buenos Aires 2008 slide 35 Conclusions Observational Influence pattern would provide information on different observation system New observation system Special observing field campaign Thinning is mainly performed to reduce the spatial correlation but also to reduce the analysis computational cost Knowledge of the observations influence helps in selecting appropriate data density Diagnose the impact of improved physics representation in the linearized forecast model in terms of observation influence

36. ECMWF Observation Influence DA School Buenos Aires 2008 slide 36 Background and Observation Tuning in ECMWF 4D-Var Observations Model

37. ECMWF Observation Influence DA School Buenos Aires 2008 slide 37 Influence Matrix Computation B A sample of N=50 random vectors from (0,1) Truncated eigenvector expansion with vectors obtained through the combined Lanczos/conjugate algorithm. M=40

38. ECMWF Observation Influence DA School Buenos Aires 2008 slide 38 Hessian Approximation B-A 500 random vector to represent B

39. ECMWF Observation Influence DA School Buenos Aires 2008 slide 39 Ill-Condition Problem A set of linear equation is said to be ill-conditioned if small variations in X=(HK I-HK) have large effect on the exact solution ŷ, e.g matrix close to singularity A Ill-conditioning has effects on the stability and solution accuracy. A measure of ill-conditioning is A different form of ill-conditioning can results from collinearity: XX T close to singularity Large difference between the background and observation error standard deviation and high dimension matrix

40. ECMWF Observation Influence DA School Buenos Aires 2008 slide 40 Flow Dependent b : MAM T +Q DRIBU ps Influence