1. Page 1© Crown copyright 2005 Numerical space weather prediction: can meteorologists forecast the way ahead? Dr Mike Keil, Dr Richard Swinbank and Dr Andrew Bushell ESWW, November 2005

2. Page 2© Crown copyright 2005 Introduction What National Met Services (NMS) do and how does this fit in with Space Weather? How did they get there? What can be learned for Numerical Space Weather Prediction? What does the future hold?

3. Page 3© Crown copyright 2005 National Met Services What do they do?

4. Page 4© Crown copyright 2005 How’s it done? Numerical Weather Prediction Model Observations Analysis data assimilation Forecast T+1T+2T+3T+4T+5T+6T+12T+24T+48T+…

5. Page 5© Crown copyright 2005 Development of NWP: Vilhelm Bjerknes (1862-1951) had a vision! L.F. Richardson’s first forecast sometime between 1916 and 1918. 1950 Charney ran the first forecast on a computer It took longer to subjectively quantify the ICs than run the forecasts! So far, no mention of Data Assimilation… Clearly a need for an objective way of specifying the initial conditions and analysis

6. Page 6© Crown copyright 2005 Development of DA: 1949 Panofski had been creating objective analysis using interpolation techniques 1954 Gilchrist and Cressman had two ideas: numerical forecasts as a source of background info automatic quality control of data 1955 Bergothorsson and Doos – analyse observation increments 1961 Thompson – use DA to propagate info into data voids

7. Page 7© Crown copyright 2005 NWP in the present day: State Time Corrected forecast Initial forecast T+0T+6 Observations Development of NWP models and increased computer performance has led to more sophisticated assimilation schemes

8. Page 8© Crown copyright 2005 The virtuous cycle observations assimilation modelling science

9. Page 9© Crown copyright 2005 1953 Storm

10. Page 10© Crown copyright 2005 The virtuous cycle observations assimilation modelling science

11. Page 11© Crown copyright 2005 Lessons from Numerical Weather Prediction  Data Assimilation combines information from observations with a background state.  The background state could come from a number of sources: subjective analysis, climatological averages, empirical models  To exploit the full potential of data assimilation, the background state should be produced using a physically-based numerical model.  This should be the approach to follow for SW assimilation

12. Page 12© Crown copyright 2005 Lessons from Numerical Weather Prediction  A physically-based numerical model is not just required for data assimilation.  A physically-based model is an essential part in fully establishing the virtuous cycle.  Empirical models can serve a useful purpose; however their potential for development is restricted.  Physically-based models provide a route for long-term space weather scientific growth

13. Page 13© Crown copyright 2005 Lessons – data issues  Satellite data is the most obvious crossover area  Co-ordination is handled by WMO  Global Observing System  info / education / transition  Most NMS assimilate data from around 25 operational satellites  What about experimental satellites?  WMO set the “rules”  GTS infrastructure NMS have experience in handling and processing vast amounts of data

14. Page 14© Crown copyright 2005 Lessons – common data sources GPS RO observations is a good example Mid-90s humidity and temperature profiles from GPS Realistic assimilation first carried out at the Met Office Operational use next year Techniques can be applied to assimilate TEC COSMIC: Constellation Observing System for Meteorology, Ionosphere and Climate 6 space craft – provide TEC, allow operational monitoring Data available in near real time for scientific research

15. Page 15© Crown copyright 2005 Lessons – other questions along the way There are issues relevant to SW that have already been tackled by the met community:  Bias correction of data  Assimilation of derived products or raw values?  Pain before the gain – increasing complexity  Potential for development  Timeliness of data  Ensembles

16. Page 16© Crown copyright 2005 The future: operational met models Most operational met models are pushing beyond the stratosphere  Why?  Met Office global model will have a lid at 63km  Research model with a 86km lid  Other centres go higher – eg CMAM 210km  Sensible to have a joined-up approach to common issues

17. Page 17© Crown copyright 2005 The future: scientific collaboration The Met Office are interested in Space Weather science! Potential areas of research:  Coupling between weather and space weather models  Lower boundary forcings?  Upwards/downwards control?  Fully coupled models (whole atmosphere approach)?  Applying data assimilation expertise to space weather assimilation  Radio occultation assimilation experience  Funding

18. Page 18© Crown copyright 2005 The future: numerical space weather prediction Within a decade (?) there will be a requirement for operational numerical space weather prediction  Why? Primarily military with commercial applications  How?  Following the framework used in operational NWP  Learning from met experience in key areas  Utilising the facilities of NMS eg supercomputers, observation supply, 24/7 capabilities, down-stream dissemination to end users  This way of working already exists in operational oceanography at the Met Office

19. Page 19© Crown copyright 2005 Conclusions  The development over many years of NWP presents a framework for Numerical Space Weather Prediction  Fully establish the “virtuous cycle” for SW  Some pain can be avoided by learning from the met community!  Science can be pushed forward through collaboration  Operational Space Weather within a decade?  National Met Services offer crucial facilities  Successful partnerships of this kind already exist  Thanks for listening!

20. Page 20© Crown copyright 2005 Questions

21. Page 21© Crown copyright 2005 The framework of modern DA: Analysis Model Observations data assimilation Forecast T+1T+2T+3T+4T+5T+6 Bjerknes / Richardson / Charney Panofski Gilchrist and Cressman Thompson Bergothorsson and Doos

22. Page 22© Crown copyright 2005 DA: hierarchy  Most assimilation schemes operate sequentially.  As long as the evolution of errors is close to linear, an extended Kalman filter is the optimum statistical assimilation method.  Hierarchy of different approximations to the Kalman filter:  Direct insertion  Nudging  Statistical interpolation  3D-variational (old Met Office system)  4D-variational (current Met Office global system)  Ensemble Kalman Filter  Choose appropriate level of complexity / cost.

23. Page 23© Crown copyright 2005 DA: cost function  Analysis is found by minimising a cost function quantifying misfit between model fields x and both obs y o and background x b Where y=H(x) is a prediction of y o  In 3D-VAR, the analysis is calculated using observations at one particular time  In 4D-VAR, the analysis uses observations at their correct validity time  Met Office system written in incremental form

24. Page 24© Crown copyright 2005 DA: 4D Var  4D-VAR uses observations over a given time window.  Allows use of observations at correct time, and exploit information in a time sequence.  Requires use of a (simplified, linear) model and its adjoint. (Not clear what level of simplification is appropriate for ionosphere).

25. Page 25© Crown copyright 2005 DA: summary  3DVAR (with a 6h assimilation cycle for global model) currently used for the stratospheric version of the UM. 4D VAR in the global model was implemented during 2004.  Main advantage of 4DVAR is use of observations at correct time / use of time sequence of obs. Requires adjoint of forecast model. (3DVAR only requires adjoint of observation operator.)  Kalman Filter updates error covariance as well as state – much more expensive; requires drastic simplification.