ECMWF Observation Influence Training Course 2010 slide 1 Influence matrix diagnostic to monitor the assimilation system Carla Cardinali

ECMWF Observation Influence Training Course 2010 slide 2 Monitoring Assimilation System ECMWF 4D-Var system handles a large variety of space and surface- based observations. It combines observations and atmospheric state a priori information by using a linearized and non-linear forecast model Effective monitoring of a such complex system with 10 8 degree of freedom and 10 7 observations is a necessity. No just few indicators but a more complex set of measures to answer questions like  How much influent are the observations in the analysis?  How much influence is given to the a priori information?  How much does the estimate depend on one single influential obs?

ECMWF Observation Influence Training Course 2010 slide 3 Influence Matrix: Introduction Unusual or influential data points are not necessarily bad observations but they may contain some of most interesting sample information In Ordinary Least-Square the information is quantitatively available in the Influence Matrix Tuckey 63, Hoaglin and Welsch 78, Velleman and Welsch 81 Diagnostic methods are available for monitoring multiple regression analysis to provide protection against distortion by anomalous data

ECMWF Observation Influence Training Course 2010 slide 4 Influence Matrix in OLS The OLS regression model is Y (mx1) observation vector X (mxq) predictors matrix, full rank q β (qx1) unknown parameters  (mx1) error The fitted response is OLS provide the solution m>q

ECMWF Observation Influence Training Course 2010 slide 5 Influence Matrix Properties S (mxm) symmetric, idempotent and positive definite matrix It is seen The diagonal element satisfy Cross-SensitivitySelf-Sensitivity Average Self-Sensitivity=q/m

ECMWF Observation Influence Training Course 2010 slide 6 Influence Matrix Related Findings The change in the estimate that occur when the i-th is deleted CV score can be computed by relying on the all data estimate ŷ and S ii

ECMWF Observation Influence Training Course 2010 slide 7 Outline Conclusion Observation and background Influence Generalized Least Square method Findings related to data influence and information content Toy model: 2 observations

ECMWF Observation Influence Training Course 2010 slide 8 Hx b y HK I-HK Hx b y HK I-HK Solution in the Observation Space The analysis projected at the observation location The estimation ŷ is a weighted mean Hx b y HKI-HK B(q x q)=Var(x b ) R(p x p)=Var(y) K(q x p) gain matrix H(p x q) Jacobian matrix

ECMWF Observation Influence Training Course 2010 slide 9 Influence Matrix Observation Influence is complementary to Background Influence

ECMWF Observation Influence Training Course 2010 slide 10 Influence Matrix Properties The diagonal element satisfy

ECMWF Observation Influence Training Course 2010 slide 11 Synop Surface Pressure Influence >1 S ii >1 due to the numerical approssimation

ECMWF Observation Influence Training Course 2010 slide 12 Aircraft 250 hPa U-Comp Influence S ii >1 due to the numerical approssimation >1

ECMWF Observation Influence Training Course 2010 slide 13 QuikSCAT U-Comp Influence >1 S ii >1 due to the numerical approssimation

ECMWF Observation Influence Training Course 2010 slide 14 Observation Influence: Vertical levels >1 S ii >1 due to the numerical approssimation

ECMWF Observation Influence Training Course 2010 slide 15 AMSU-A channel 13 Influence S ii >1 due to the numerical approssimation >1

ECMWF Observation Influence Training Course 2010 slide 16 Toy Model: 2 Observations x1x1 x2x2 y2y2 y1y1 Find the expression for S as function of r and the expression of for α=0 and 1 given the assumptions:

ECMWF Observation Influence Training Course 2010 slide 17 Toy Model: 2 Observations x1x1 x2x2 y2y2 y1y1 S ii r 1 1 1/2 1/3 1/2  =1  =0 0

ECMWF Observation Influence Training Course 2010 slide 18 Consideration (1) Where observations are dense S ii tends to be small and the background sensitivities tend to be large and also the surrounding observations have large influence (off- diagonal term) When observations are sparse S ii and the background sensitivity are determined by their relative accuracies (r) and the surrounding observations have small influence (off-diagonal term)

ECMWF Observation Influence Training Course 2010 slide 19 Toy Model: 2 Observations  =0 =1 x1x1 x2x2 y2y2 y1y1

ECMWF Observation Influence Training Course 2010 slide 20 Consideration (2) When observation and background have similar accuracies (r), the estimate ŷ 1 depends on y 1 and x 1 and an additional term due to the second observation. We see that if R is diagonal the observational contribution is devaluated with respect to the background because a group of correlated background values count more than the single observation (2-α 2 → 2). Also by increasing background correlation, the nearby observation and background have a larger contribution

ECMWF Observation Influence Training Course 2010 slide 21 Global and Partial Influence Global Influence = GI =   100% only Obs Influence 0% only Model Influence Partial Influence = PI = Type Variable Area Level

ECMWF Observation Influence Training Course 2010 slide 22 Global and Partial Influence Level Type SYNOP AIREP SATOB DRIBU TEMP PILOT AMSUA HIRS SSMI GOES METEOSAT QuikSCAT Variable uvTqpsTbuvTqpsTb Area TropicsS.HemN.Hem EuropeUSN.Atl …

ECMWF Observation Influence Training Course 2010 slide 23 Average Influence and Information Content Global Observation Influence GI=15% Global Background Influence I-GI=85% 2003 ECMWF Operational

N.Hemisphere PI = 15% Tropics PI = 17.5% S.Hemisphere PI = 12% GI = 15.3% 2003 ECMWF Operational

ECMWF Observation Influence Training Course 2010 slide ECMWF Op DFS 20%

ECMWF Observation Influence Training Course 2010 slide 26 DFS % first bar Global, second North Pole and third South Pole Global Information Content 20% North Pole 5.6% South Pole 1.1%

ECMWF Observation Influence Training Course 2010 slide 27 DFS % Global Information Content 20% North Pole 5.6% South Pole 1.1% Tropics 5.5%

ECMWF Observation Influence Training Course 2010 slide 28 Evolution of the B matrix: B computed from EnDA X t + ε Stochastics y+ε o SST+ε SST Xb+εbXb+εb y+ε o SST+ε SST AMSU-A ch6

ECMWF Observation Influence Training Course 2010 slide 29 Evolution of the GOS: Interim Reanalysis Aircraft hPa

ECMWF Observation Influence Training Course 2010 slide 30 Evolution of the GOS: Interim Reanalysis AMSU-A ch

ECMWF Observation Influence Training Course 2010 slide 31 Evolution of the GOS and of the B: Interim Reanalysis AMSU-A ch

ECMWF Observation Influence Training Course 2010 slide 32 Evolution of the GOS: Interim Reanalysis AMSU-A

ECMWF Observation Influence Training Course 2010 slide 33 Evolution of the GOS: Interim Reanalysis U-comp Aircraft, Radiosonde, Vertical Profiler, AMV

ECMWF Observation Influence Training Course 2010 slide 34 Conclusions The Influence Matrix is well-known in multi-variate linear regression. It is used to identify influential data. Influence patterns are not part of the estimates of the model but rather are part of the conditions under which the model is estimated Disproportionate influence can be due to:  incorrect data (quality control)  legitimately extreme observations occurrence  to which extent the estimate depends on these data S ii =1 Data-sparse  Single observation Model under-confident (1-S ii ) S ii =0 Data-dense Model over-confident  tuning (1-S ii )

ECMWF Observation Influence Training Course 2010 slide 35 Conclusions Observational Influence pattern would provide information on different observation system  New observation system  Special observing field campaign Thinning is mainly performed to reduce the spatial correlation but also to reduce the analysis computational cost  Knowledge of the observations influence helps in selecting appropriate data density Diagnose the impact of improved physics representation in the linearized forecast model in terms of observation influence

ECMWF Observation Influence Training Course 2010 slide 36 Background and Observation Tuning in ECMWF 4D-Var Observations Model

ECMWF Observation Influence Training Course 2010 slide 37 Influence Matrix Computation B A sample of N=50 random vectors from  (0,1) Truncated eigenvector expansion with vectors obtained through the combined Lanczos/conjugate algorithm. M=40

ECMWF Observation Influence Training Course 2010 slide 38 Hessian Approximation  B-A 500 random vector to represent B

ECMWF Observation Influence Training Course 2010 slide 39 Ill-Condition Problem A set of linear equation is said to be ill-conditioned if small variations in X=(HK I-HK) have large effect on the exact solution ŷ, e.g matrix close to singularity A Ill-conditioning has effects on the stability and solution accuracy. A measure of ill-conditioning is A different form of ill-conditioning can results from collinearity: XX T close to singularity Large difference between the background and observation error standard deviation and high dimension matrix

ECMWF Observation Influence Training Course 2010 slide 40 Flow Dependent  b : MAM T +Q DRIBU ps Influence