1. Testing the assimilation of highly sampled aircraft wind observations (GADS) in the ECMWF global model
Workshop on Wind Profiles and Mesoscale Data Assimilation, Ljubljana, September 2016
Michael Rennie, ECMWF
In collaboration with Joel Tenenbaum (State University of New York (SUNY)) and Lars Isaksen (ECMWF)

2. Outline What is GADS? Assimilation tests and initial OSEs
OSEs with improved QC
Latest OSE results

3. What is GADS? Global Aircraft Data Set
Flight data recordings of British Airways Boeing 747 and 777 aircraft
Long-haul flights
Most data at cruise level – but also ascent/descent
Data available mostly in Northern Hemisphere, but some flights to Australia, S. America and S. Africa
Different modes:
Mode B: Wind reported every 4 s (~0.9 km horizontal sampling)
Mode C: every 32 s (~7 km)
82% is mode B data
Provides wind and temperature “point” observations
< 4 km res., 1 m/s accuracy, see Rukhovets and Tenenbaum (1998)
With “pressure altitude”, lat, lon, date/time
Basically the same observation method as Aircraft Meteorological DAta Relay (AMDAR)
but AMDAR typical sampling ~100 km

4. Where is the data from?
Provided by Joel Tenenbaum (SUNY) to ECMWF for a scientific research collaboration project
Only a limited subset is available to ECMWF
December 2010, January and February 2011 provided
Data is in a bespoke ASCII format
Geolocation, temperature, wind speed, wind direction, turbulence information
Investigations have focussed on using wind observations from GADS
Test impact of assimilating GADS without any thinning or superobbing
Assess what scales the model can represent

5. Section of example flight: GADS wind is provided as speed and direction
50 km
Can see rounding precision error in wind speed (nearest knot)
Wind speed / ms-1
Wind variability at small horizontal scales evident:
thought to be real wind variability, rather than noise
To the nearest knot

6. Wind direction
370
Wind direction/ degrees

7. GADS: example coverage in 12 hour period
Typically ~125,000 records (u, v and T) in a 12 hours period
Very high density, but only BA flight paths

8. Try assimilating GADS winds as if Aeolus HLOS winds
Convert GADS winds to HLOS wind
u-component to HLOS wind in +ve u direction
Technically easier to use as-if Aeolus data
Similar thing done with conventional winds in e.g. Horanyi et al. 2014
Observing System Experiment:
Control: Normal IFS setup
Experiment: control + assimilate GADS HLOS wind (zonal)
Jan-Feb 2011 dataset
No thinning or superobbing of GADS
4D-Var, global
Climatological B-matrix; EDA first implemented in June 2011

9. Typical vertical sampling with GADS as zonal HLOS wind
during 4D-Var window

10. Typical O-B of GADS zonal HLOS wind
m/s

11. Need for bespoke QC of GADS data
Early experiments gave negative impact
After investigating found some gross errors in dataset e.g. values out of physical range, wrong date-times, spurious repeated values
So implemented a careful pre-assimilation QC of the data

12. OSE in late 2015: GADS OSE ECMWF CY42R1 T511 outer loop (~40 km grid)
T255 (~80 km grid) final of three inner loops (analysis increments)
Bespoke GADS QC applied to reject any suspicious data; prior to creation of ODB
Also, reject observations with p > 500 hPa
since often find spurious data recorded when aircraft on ground
Assigned observation error 2.5 m/s – similar weight given to AMDAR
Global aircraft dataset, BA long-haul flights

13. Example GADS assimilation results DA during window for cycle 2011/01/04 00Z
~400 km
Analysis pulls strongly to GADS, but doesn't fit small-scale variability e.g. < 100 km
16/05/ L2B/C PM31 L2B/C update
during 4D-Var window

14. Example: jet stream maximum wind speed underestimated in background forecast
~400 km
~10 m/s
during 4D-Var window

15. Example GADS data with high wind variability
~400 km
~400 km
during 4D-Var window

16. 2.5 m/s OSE: Comparing “normal” aircraft to GADS. O-B, O-A statistics
Northern Hemisphere stats
O-A
“Normal” aircraft u-wind
GADS O-B st. dev. is similar to “normal” aircraft
Analysis pulls more strongly to GADS than “normal” aircraft
m/s
GADS HLOS wind (in u dir.)
m/s

17. 2.5 m/s OSE: fit to other observations; experiment-control; O-B, O-A std. dev.
GPSRO
AMSU-A
O-A
O-B
Statistically significant degradation in fit of background forecast and analysis to other observation types
Not good!
Conventional u-wind, not including GADS

18. 2.5 m/s OSE: Forecast impact of GADS
Change in RMSE (verified against own analysis), red is worse
Wind velocity
Very large change in analysis in NH
Little statistical significance in longer range i.e. neutral

19. 2.5 m/s GADS OSE summary
GADS with 2.5 m/s assigned obs error makes substantial changes to the analysis in Northern Hemisphere
Large degradation in O-Bs compared to other important observations seen
Pulling strongly to GADS data; noisy analysis in the vicinity
Neutral impact at longer-range in 2 month OSE
Why negative impact?
Suspect too much weight given to GADS, perhaps due to not accounting for correlated representativeness error and/or correlated instrument error

20. Recent results: GADS OSE in Sep. 2016
Same settings as before; apart from:
ECMWF CY43R1
TcO399 outer loop; ~28 km grid
T255 inner loop (analysis increment grid); ~80 km grid
Try again with significantly less weight given to GADS
Assign inflated obs error; 10 m/s to compensate for correlated errors

21. 10 m/s OSE: Comparing “normal” aircraft to GADS. O-B, O-A statistics
Northern Hemisphere stats
O-A
“Normal” aircraft u-wind
GADS O-B st. dev. is similar to “normal” aircraft
Still pulls to closely to GADS but less so < 300 hPa
m/s
GADS HLOS wind (in u dir.)
m/s

22. 2.5 m/s OSE: Comparing “normal” aircraft to GADS. O-B, O-A statistics
Northern Hemisphere stats
O-A
“Normal” aircraft u-wind
m/s
GADS HLOS wind (in u dir.)
m/s

23. 10 m/s OSE: fit to other observations; experiment-control; O-B, O-A std. dev.
GPSRO
AMSU-A
O-A
O-B
Conventional wind
Fairly neutral impact here; much better than 2.5 m/s OSE

24. 10 m/s OSE: Further obs types
Some improvements in fit to other observations in O-B statistics
Radiosondes T
Radiosondes q
AMVs
SSMIS

25. 2016 OSE: Forecast impact of GADS
Change in RMSE (verified against own analysis), red is worse, blue is good
Geopotential height
Wind velocity
Neutral
Need longer period
Don’t see overfitting effect at short FC range

26. Conclusions from GADS OSEs (so far)
Bespoke QC of GADS is critical to avoid some gross errors
4D-Var appears to handle dense dataset well
Is not trivial to get positive impact from densely sampled and (apparently) high accuracy dataset
Assume because of limited geographical coverage over already well observed flight paths and using incorrect R matrix
With inflated obs error (10 m/s) get neutral/slightly positive impact
Slightly positive against some other obs types
Mitigating correlated representativeness error (or even correlated instrument error)?
ECMWF OSEs typically need several months to get stat. significant results
Just started experiments with modifications in hope of getting a stronger signal:
Keep 10 m/s assigned obs error
Both u and v wind components  HLOS winds
Use 3 rather than 2 month OSE, Dec 2010 – Feb 2011, 47M obs

27. Any questions?

28. Assessment of 2014 GADS OSE results
Analysis creates reasonable looking wind structures near the GADS data at >100 km scales
Issue: O-B std. dev. ~4-5 m/s for GADS
AMDAR O-B st. dev. ~2-3 m/s. Suggests problem with GADS, since same basic observation as GADS
GADS
O-A
O-B
m/s
m/s

29. 2014 OSE impact results Impact from GADS was v. negative
Reason: GADS O-B statistics significantly worse than AMDAR data
Should be similar, since effectively the same observations just sampled a lot more!
On closer inspection of O-B results
Turns out there some erroneous data e.g. values out of physical range, wrong date- times, spurious repeated values
Hence worked on bespoke pre-assimilation QC to select only the valid data – this was a lot of work!

30. Late 2015 OSE: Forecast impact of GADS
Change in RMSE (verified against own analysis), red is worse
Geopotential height
Wind velocity
Very large change in analysis in NH
Little statistical significance in longer range i.e. neutral

31. Change in RMSE vs forecast range
Geopotential height
Vector wind