1. Impact of Observations – recent studies Jean-Noël Thépaut credits: ECMWF staff, including Tony McNally, Stephen English, Alan Geer, Cristina Lupu
(strong NWP and DA bias…)
(largely inspired from a recent seminar at ECMWF  Tony McNally)

2. Overview What do we mean by impact (analysis / forecasts)
What factors influence impact ?
What diagnostics are available to measure impact and are they reliable ?
An assessment of current satellite impact on NWP forecasts
Final note on the forthcoming OSE/OSSE workshop

3. WMO Integrated Global Observing System
Courtesy: WMO
ECMWF Preview – DTS
TWP - ICE
Supported by field campaign experiments,
Data targeting studies,
etc.
WMO plays a crucial role of course to coordinate, integrate and sustain the global observing system, with a number of commissions, expert teams and processes in place to ensure that the design of the global observing system has a strong bottom-up dimension. And in addition, the science is supported by a number of continual field campaign experiment, data targetting studies, etc.

4. (big Volume, big Velocity, big Variety)
5.5 more instruments per year
THIS IS BIG DATA!
(big Volume, big Velocity, big Variety)
This plot represents the evolution of the number of satellite data products monitored at ecmwf for the past 18 years. Obviously the days where one specialist looked after assimilation from 1-2 instruments are long since over… The volumes involved are not necessarily linear but what is clear anyhow is that we are talking big data here, with big data volumes, big velocity in terms of retrieving and archiving requirements, and big variety of products

5. Cloud Radar Reflectivity
Models and observation operators have become much more realistic and accurate
CLOUDSAT
Continual improvement of ECMWF short-range precipitation forecasts with respect to ground-based radar data.
Cloud Radar Reflectivity
These improvements open new opportunities to explore more observations
A second and third examples show on the left the model representation of cloud radar reflectivity and the associated cloudsat measurements. Here again the similarity is becoming excellent, and on the right, the plot shows the improvement of short-range precipitation forecasts over the last twelve years as compared with a ground based radar network (NEXRAD) over the US. Being able to depict so accurately observables for quantities such as moisture, clouds an precips is of course relevant for extreme weather prediction

6. Overview What do we mean by impact (analysis / forecasts)
What factors influence impact ?
What diagnostics are available to measure impact and are they reliable ?
An assessment of current satellite impact on NWP forecasts

7. What do we mean by impact ?
Traditionally:
Range (12hrs, 5 days, 10 days…)
Parameter (gp height, wind, temperature, humidty …)
Altitude (surface, 500hPa, 1hPa)
Region (global, NH, SH, Tropics, Europe)
Note: Observation impact has a sign ….it can be good and bad!

8. Overview What do we mean by impact (analysis / forecasts)
What factors influence impact ?
What diagnostics are available to measure impact and are they reliable ?
An assessment of current satellite impact on NWP forecasts
How satellite data impact model development

9. Factors that determine impact ?
Observation quality
Observed quantity (important ? already known?)
Observation usability (ambiguity)
Observation spatial coverage
Observation time
Tuning of the assimilation system (correct specification of B, R, BC, QC)

10. Factors that determine impact ?
Observation quality
Observed quantity (important ? already known?)
Observation usability (ambiguity)
Observation spatial coverage
Observation time
Tuning of the assimilation system (correct specification of B, R, BC, QC)

11. Mechanism: Dynamical adjustments in 4DVAR
Assimilation of a single 183 ±1 GHz observation
Start
Time of
observation
Assimilation window
UTH
increment
( hPa mean RH)
Humidity reduction at observation time generated by changes in wind (and other dynamical variables) 1000km away, 9h earlier!
Zonal wind
increment
at 400 hPa

12. No impact of USPS over USA !

13. Figure 2. Orbits of the EPS and USPS in the 00z (left) and 12z (right) 4DVAR 12 hour assimilation window. Those coloured red (for USPS) and blue (for EPS) are observed in the lattermost 3 hours of the assimilation window.

14. Factors that determine impact ?
Observation quality
Observed quantity (important ? already known?)
Observation usability (ambiguity)
Observation spatial coverage
Observation time
Tuning of the assimilation system (correct specification of B, R, BC, QC)

15. Observation error specification: Impact on FG-departures for other observations
AMSU-A, tropics
Radiosondes - T, tropics
GPSRO, global
IASI inter-channel error correlation matrix
A quick illustration here to show that observation error specification is extremely important. This is a study showing the impact of taking into account the error correlation between channels from the IASI interferrometer onboard METOP, (left panel) on the fit of the model first-guess to other independent observations. This impact is significant and general.
< 100% :
improved model first-guess fit to observations

16. Sandy: impact of background error specifications
No Polar: EDA-based background error covariances
Control
No Polar: Brute force
Sandy: impact of background error specifications
If we withdraw the observations without informing the asimilation system, the 5 and 4 day forecasts become very poor (admittedly a very gross experiment). Point to illustrate, is that by specifying the appropriate background error, the system recovers partly the forecast by making a better use of other observations
Four day forecasts of surface pressure launched from 26th October (left) and five day forecasts from the 25th October (right) for the control (grey), NOPOLAR (red) and NOPOLAR-EDA (blue). Contours at 10hPa intervals with shading below 970hPa).

17. Overview What do we mean by impact (analysis / forecasts)
What factors influence impact ?
What diagnostics are available to measure impact and are they reliable ?
An assessment of current satellite impact on NWP forecasts

18. Diagnostics Available
Observing System Experiments (OSE)
Denial or addition experiments
Periodic statistical evaluations
Case studies
Adjoint Sensitivity Diagnostics (ASD)
Impact assessed without denial
Case studies ?

19. Pros and Cons of OSE
Extremely (prohibitively?) expensive to run long periods (needed for small signals)
Adding or denying a data type may require background errors to be retuned*
Verifying short-range forecasts is less reliable
The only measure of medium-range observation impact
They give the only clear definitive answer to the question “what if I did not have this satellite ?”

20. Diagnostics Available
Observing System Experiments (OSE)
Denial or addition experiments
Periodic statistical evaluations
Case studies
Adjoint sensitivity Diagnostics (ASD)
Impact assessed without denial
Case studies ?

21. Forecast impact of all-sky assimilation of microwave WV Compared to clear-sky assimilation
Change in vector wind RMS error, NH, 500 hPa
SH Tropics NH
All-sky MW WV – No MW WV
Clear-sky MW WV – No MW WV

22. OSEs for surface-sensitive MW sounder data over land/sea-ice
Experiments over 8 months:
2 June – 30 Sept 2014; 2 Dec 2014 – 31 March 2015
Base: No surface-sensitive MW sounder data over land and sea-ice, otherwise as operations
Base + MW seaice: Add surface-sensitive MW sounders over sea-ice
Base + MW seaice + MW WV land: Add MW humidity sounders over land
Base + WV seaice + MW land: Add surface-sensitive MW temperature sounders over land (ie, data usage as in operations)
MW humidity sounder channels:
ATMS ch18-22 (clear-sky) – 1 instrument
MHS ch 3-5 (all-sky) – 4 instruments
SSMIS ch 9-11 (all-sky) – 1 instrument
Surface-sensitive MW temperature sounder channels:
ATMS ch 6-8 (clear-sky) - 1 instrument
AMSU-A ch 5-7 (clear-sky) - 5 instruments

23. Impact of surface-sensitive MW sounders over sea-ice and land, respectively
Base + MW seaice + MW land
Base + MW seaice
vs Base

24. Impact of surface-sensitive MW humidity and temperature sounders over land
Base + MW seaice + MW land
Base + MW seaice + MW WV land
vs Base + MW seaice

25. Giovanna De Chiara

26. Impact of observations in coupled systems: A case study
Laloyaux et al 2015b
Tropical Cyclone Phailin, Bay of Bengal
Assimilation of scatterometer wind data

27. Impact of observations in coupled systems: A case study
Laloyaux et al 2015b
Tropical Cyclone Phailin, Bay of Bengal
Assimilation of scatterometer wind data
Verification against Argo data
Ocean temperature at 40m depth: Impact of surface wind observations
uncoupled
coupled 2K
scatt impact
Argo
Time series of ocean temperature observations at a depth of 40 metres by the Argo float (Observations) with (a) the temperature analyses produced by the CERA system with scatterometer data assimilation (CERA-SCATT) and without scatterometer data assimilation (CERA-NOSCATT); and (b) the temperature analyses produced by the UNCPL system with scatterometer data assimilation (UNCPL-SCATT) and without scatterometer data assimilation (UNCPL-NOSCATT).
5 days

28. Diagnostics Available
Observing System Experiments (OSE)
Denial or addition experiments
Periodic statistical evaluations
Case studies
Adjoint Sensitivity Diagnostics (ASD)
Impact assessed without denial
Case studies ?

29. Adjoint Sensitivity Diagnostics
E(36hr)
E(24hr)
adjoint of forecast model

30. Pros and Cons of ASD
Can only operate a short-range where verification is least reliable
Problems relating TE metric to parameters and consistency of adjoint model (dry/wet)
Poor observation error tuning can produce misleading results*
Very affordable (compared to OSE)
Allows detailed evaluation of individual channel impacts

31. Forecast sensitivity to observations (Oct 2015)
(mostly all-sky)
(all-sky)

32. The WMO 6th Workshop on the Impact of Various Observing Systems on NWP Shanghai, China May
organised by the Inter Programme Expert Team on the Observing System Design and Evolution (IPET-OSDE)
Will review recent changes and trends:
changes and developments have affected the global observing system
more efforts have been devoted to meso-scale observing and assimilation systems.
Trend toward using techniques other than OSEs to document data impact, such as adjoint- and ensemble-based forecast sensitivity observation impact (FSOI and EFSOI)
estimates of analysis uncertainty.