1. Forecasts & their Errors Ross Bannister 7th October 2008 1 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Forecasts & their Errrors Ross Bannister National Centre for Earth Observation (the Data Assimilation Research Centre) Thanks to: Stefano Migliorini (NCEO), Mark Dixon (MetO), Mike Cullen (MetO), Roger Brugge (NCEO) Forecast Possible error in forecast Horiz. winds and pressure, at 5.5 km Met Office North Atlantic/European LAM

2. Forecasts & their Errors Ross Bannister 7th October 2008 2 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Forecast errors Forecast errors (from a numerical model):  are a fact of life!  depend upon the model formulation,  synoptic situation (‘flow dependent’),  model’s initial conditions,  length of the forecast.  are impossible to calculate in reality, δx = x f - x t. Of interest:  forecast error statistics - the probability density fn. of x t, P f (x t ). Applications:  probabilistic forecasting.  model evaluation/monitoring.  state estimation (data assimilation).

3. Forecasts & their Errors Ross Bannister 7th October 2008 3 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Seminar structure  Probability density functions (PDFs) of the state, P f (x t ).  The use of P f (x t ) in data assimilation problems.  Measuring P f (x t ).  Modelling P f (x t ) for large-scale data assimilation.  Refining P f (x t ) for large-scale data assimilation.  Challenges for small-scale Meteorology.

4. Forecasts & their Errors Ross Bannister 7th October 2008 4 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges PDF of state, P f (x t ) P f (x t ) xtxt xfxf Impossible state Probable state Possible but unlikely state 0 Forecast comprising a single number σ = √var(δx) x f = x t + δx

5. Forecasts & their Errors Ross Bannister 7th October 2008 5 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Two-component state vector Forecast comprising two numbers Pf(xt)Pf(xt) 0 x f = x t + δx σ x1 = √var(δx 1 ) σ x2 = √var(δx 2 ) cov( δx 1,δx 2 )

6. Forecasts & their Errors Ross Bannister 7th October 2008 6 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Geophysical error covariances – B The B-matrix  specifies the PDF of errors in x f (Gaussianity assumed)  describes the uncertainty of each component of x f and  how errors of elements in x f are correlated  is important in data assimilation problems 10 7 – 10 8 elements structure function associated (e.g.) with pressure at a location δu δv δp δT δq x f = u –– v –– p –– T –– q B =

7. Forecasts & their Errors Ross Bannister 7th October 2008 7 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Example standard deviations (square- root of variances) From Ingleby (2001)

8. Forecasts & their Errors Ross Bannister 7th October 2008 8 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Example geophysical structure functions (covariances with a fixed point) Univariate structure function Multivariate structure functions

9. Forecasts & their Errors Ross Bannister 7th October 2008 9 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Covariances are time dependent Structure function for tracer in simple transport model 1.0 0.9 0.7 0.8 t = 0 t > 0 1.0 0.9 0.7 0.8

10. Forecasts & their Errors Ross Bannister 7th October 2008 10 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges P f (x t ) \ B in data assimilation Data assimilation combines the PDFs of i.forecast(s) from a dynamical model, P f (x t ) and ii.measurements, P ob (y|x t ) to allow an ‘optimal estimate’ to be found (Bayes’ Theorem). Maximum likelihood solution (Gaussian PDFs) forecast = prior knowledge Solved e.g. by direct inversion or by variational methods PDF of combination of forecast and observational information

11. Forecasts & their Errors Ross Bannister 7th October 2008 11 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Pseudo satellite tracks Tracer assimilation Data assimilation example (for inferred quantities) x(0) initial conditions y(t 1 ) y(t 2 ) y(t 3 ) y(t 4 ) y(t 5 ) T, q, O 3 satellite radiances Initial conditions inferred from measurements made at a later time Sources/sinks of tracer, rmeasurements of tracer r –– sources/ sinks Tracer + source/sink assimilation 30-day assimilation

12. Forecasts & their Errors Ross Bannister 7th October 2008 12 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Dangers of misspecifying P f (x t ) \ B in data assimilation? Example 1: Anomalous correlations of moisture across an interface Example 2: Anomalous separability of structure functions around tilted structures Normally dry air Normally moist air

13. Forecasts & their Errors Ross Bannister 7th October 2008 13 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Ensembles Measuring P f (x t ) \ B Forecast errors are impossible to measure in reality, δx = x f - x t. All proxy methods require a data assimilation system. Analysis of innovations Differences between varying length forecasts t x √2 δx Canadian ‘quick covs’ x t √2 δx t x

14. Forecasts & their Errors Ross Bannister 7th October 2008 14 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Modelling P f (x t ) \ B with transforms for data assimilation PDF in model variables 10 7 – 10 8 elements δu δv δp δT δq (multivariate) model variable control variable transform (univariate) control variable Transform to new variables that are assumed to be univariate

15. Forecasts & their Errors Ross Bannister 7th October 2008 15 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Ideas of ‘balance’ to formulate K (and hence P f (x t ) \ B) ← streamfunction (rot. wind) pert. (assume ‘balanced’) ← velocity potential (div. wind) pert. (assume ‘unbalanced’) ← residual pressure pert. (assume ‘unbalanced’) H geostrophic balance operator (δψ → δp b ) T hydrostatic balance operator (written in terms of temperature) Approach used at the ECMWF, Met Office, Meteo France, NCEP, MSC(SMC), HIRLAM, JMA, NCAR, CIRA Idea goes back to Parrish & Derber (1992) these are not the same (clash of notation!) Implied f/c error covariance matrix

16. Forecasts & their Errors Ross Bannister 7th October 2008 16 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Assumptions This formulation makes many assumptions e.g.: A.That forecast errors projected onto balanced variables are uncorrelated with those projected onto unbalanced variables. B.The rotational wind is wholly a ‘balanced’ variable (i.e. large Bu regime). C.That geostrophic and hydrostatic balances are appropriate for the motion being modelled (e.g. small Ro regime).

17. Forecasts & their Errors Ross Bannister 7th October 2008 17 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges A. ‘Non-correlation’ test latitude vertical model level

18. Forecasts & their Errors Ross Bannister 7th October 2008 18 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Modified transform B. Rotational wind is not wholly balanced Standard transform Could there be an unbalanced component of δψ? H geostrophic balance operator T hydrostatic balance operator H anti-geostrophic balance operator

19. Forecasts & their Errors Ross Bannister 7th October 2008 19 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Non-correlation test for refined model latitude vertical model level Modified transform

20. Forecasts & their Errors Ross Bannister 7th October 2008 20 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges C. Are geostrophic and hydrostatic balances always appropriate? from Berre, 2000 E.g. test for geostrophic balance

21. Forecasts & their Errors Ross Bannister 7th October 2008 21 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges What next? Hi-resolution forecasts need hi-resolution P f (x t ) \ B High impact weather! The Reading/MetO HRAA Collaboration www.met.rdg.ac.uk/~hraa  Can forecast error covariances at hi-resolution be successfully modelled with the transform approach?  What is an appropriate transform at hi-resolution?  At what scales do hydrostatic and geostrophic balance become inappropriate? There is little known theory to guide us at hi-res. → What is the structure of forecast error covariances in such cases?

22. Forecasts & their Errors Ross Bannister 7th October 2008 22 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Hi-resolution ensembles Early results from Met Office 1.5 km LAM (a MOGREPS-like system) Thanks to Mark Dixon (MetO), Stefano Migliorini (NCEO), Roger Brugge (NCEO)

23. Forecasts & their Errors Ross Bannister 7th October 2008 23 / 23 PDFsAssimilationMeasuringModellingRefiningChallenges Summary  All measurements are inaccurate and all forecasts are wrong!  Accurate knowledge of forecast uncertainty (PDF) is useful: » to allow range of possible outcomes to be predicted, » to give allowed ways that a forecast can be modified by observations (data assimilation).  For synoptic/large scales the forecast error PDF is modelled with a change of variables and balance relations.  For hi-res (convective scales) the forecast error PDF is still important but there is no formal theory to guide PDF modelling: » hydrostatic/geostrophic balance less appropriate, » non-linearity/dynamic tendencies may be more important.