1. 2013-09-11 1 Data Quality Control and Quality Monitoring Jitze van der Meulen, 2013-09-11  WMO AMDAR PANEL

2. 2013-09-11Data Quality Control and Quality Monitoring2 Workshop on Aircraft Observing System Data Management Workshop on Aircraft Observing System Data Management (2012) Recognize comparisons of aircraft observations to NWP model background fields as a critical component of Aircraft Observations Quality Control (AO QC). Consider whether or not such comparisons should be done before AO data are exchanged on the GTS. Semi-automatic near real time monitoring information such as data counts, missing data, higher than normal rejects by the assimilation system, etc. should be exchanged regularly (monthly or more frequently as required and agreed to) between designated centres and data managers (and/or producers). This could include and Alarm/Event system. Consideration should be given to the designation of centres to carry out international QC of Aircraft Observations (WMO and ICAO), possibly before insertion on the GTS, to flag the data. That distribution of ICAO automated aircraft observations on the GTS be done using WMO approved format (BUFR) with an appropriate template (similar to the AMDAR ones) for clear identification of the source of the data (ADS, MODE-S, Aircraft ID, etc.).

3. 2013-09-11Data Quality Control and Quality Monitoring3 The AMDAR Panel has identified 20 key aspects for further developing the Aircraft Observations Quality Control (AO QC); 3rd party data ADS (ICAO), other new data sources (Mode-S) Archiving (data and metadata) Delivery (level II data; also profile data for local), relation to time/place resolution) Optimization of observations data targeting (additional, for applications) data coverage (global), provision (e.g. Africa); programme extentions Developing countries, special constraints (data comm. issues) Data format (incl. resolution) Code issues (incl. data header) Data display Data access Data transfer Typical data: Atmosph. Composition data Phenomena: Icing, Turbulence, use of data (e.g. direct input, verification) Timeliness (taking into account Q/C processes) Data checking, filtering, flagging (relation with rules, M.GDPFS) Excluding aircraft (how to manage) Quality control: monitoring (availability), technics (NWP), stages (real time, off-line); flagging principles; archiving; logistics; feed back Metadata (definition, use, archive)

4. 2013-09-11Data Quality Control and Quality Monitoring4 Outline for the Definition of the Global AO DM Framework

5. 2013-09-11Data Quality Control and Quality Monitoring5 Use of NWP NWP model forecast background fields are regarded as the most appropriate references for (near) real time quality control. For operational practices it's will be necessary to evaluate these references to define the moist appropriate choices (update intervals, forecast interval), time and space interpolation techniques or algorithms. NWP background fields as defined used for references require sufficient information on its uncertainties. Traceability to objective observations is required, providing information on its uncertainties (time and place related) and possible seasonal variations or daily characteristics (daytime/night time). In particular altitude related bias behaviour is relevant.

6. 2013-09-11Data Quality Control and Quality Monitoring6 Temperature differences statistics (all observations / quarter)

7. 2013-09-11Data Quality Control and Quality Monitoring7 Background references For reference: positions: interpolation of grid point (3D) time: nearest time stamp of run time (analyses) or forecast (+nH), called background HIRLAM/HARMONIE (short range, high resolution): 00 (run), 03 (00+3H), 06 (run), 09 (06+3H), 12 (run), etc. ECMWF (medium term): 00 (run), 03 (00+3H), 06 (00+6H), 09 (00+9H), 12 (run –or- 00+12H), etc.

8. 2013-09-11Data Quality Control and Quality Monitoring8 HIRLAM/Harmonie error std

9. 2013-09-11Data Quality Control and Quality Monitoring9 Interpretation of differences Should be based on rules, defined by Int. Metrological Organizations (ICSU, BIPM) Statistical analyses first, then definition of parameter used for further interpretation and requirements Expressed in terms of uncertainty (not STD or RMSE), preferably 95% confidence interval

10. 2013-09-11Data Quality Control and Quality Monitoring10

11. 2013-09-11Data Quality Control and Quality Monitoring11 Observed wind vector Uncertainty in FF Uncertainty in V Uncertainty in U Uncertainty in F (95% conf.)

12. 2013-09-11Data Quality Control and Quality Monitoring12 Windvector difference - expressed as scalar

13. 2013-09-11Data Quality Control and Quality Monitoring13 Windvector difference (median of daily means) - expressed as scalar

14. 2013-09-11Data Quality Control and Quality Monitoring14 Profiles: requirements, detailness

15. 2013-09-11Data Quality Control and Quality Monitoring15 Obs data = processed data AMDAR sensor data processing

16. 2013-09-11Data Quality Control and Quality Monitoring16 TA bias by aircraft type

17. 2013-09-11Data Quality Control and Quality Monitoring17 TA bias by aircraft type

18. 2013-09-11Data Quality Control and Quality Monitoring18 TA bias by aircraft type

19. 2013-09-11Data Quality Control and Quality Monitoring19 TA bias by aircraft type

20. 2013-09-11Data Quality Control and Quality Monitoring20 TA bias by aircraft type

21. 2013-09-11Data Quality Control and Quality Monitoring21 Types of errors  Observations (air temperature, wind, humidity) are incorrectly measured or derived ( i.e. not confirming the required measurement uncertainties)  Incorrect position or time of observation  Incorrect encoding ( e.g. for altitude). May be reporting incorrect positions affects currently NWP most seriously, especially when reporting from (virtual) areas with few observations (data sparse areas), like over oceans and seas and especially at lower altitudes. Using NWP background fields only, usually no distinction can be made between these three types and it is assumed that the quantities are incorrectly measured only. Appropriate tools to detect horizontal and vertical positional errors are still no part of standard QC practices, although some methods are implemented (distance check between observations). The same holds for detecting inaccurate date and time stamps and latency in processing data on board ( may be Mode-S data comparisons may be useful ).

22. 2013-09-11Data Quality Control and Quality Monitoring22 Typically, some FM42 & BUFR reports show PALT, FL < pressure altitude[RWY] Typically, some reports show only FL > altitude[RWY] or only FL > 0 The altitude issue (continued) …..

23. 2013-09-11Data Quality Control and Quality Monitoring23 Pressure altitude versus altitude Delta(ISA-T)/K  PALT/m  P/hPa |  -1.4 K +1.2 K  | Bias: +0.1 K (median) U=1.3K

24. 2013-09-11Data Quality Control and Quality Monitoring24 Coverage

25. 2013-09-11Data Quality Control and Quality Monitoring25 PALT < 200 m

26. 2013-09-11Data Quality Control and Quality Monitoring26 Timeliness

27. 2013-09-11Data Quality Control and Quality Monitoring27 BUFR templates: confusion all around FM42 (only one) UD AS01 RJTD 011100 RRX AMDAR 0111 ASC JP9ZX659 3532N 13949E 011024 //// PS250 046/012 TB/ S031 333 F002 VG///= BUFR Upper air aircraft :(multiple) IUA C01 RJTD 051846 BUFR " Ý €Ë ð JP9Z4XW5}ÕH ¿ºwaTœü˜^5f?ÿô¥£UuƒQd€GQù€˜D[WóÿÿJP9Z5WX9}ÕK ±[ ucE€ŒXIuxÿÿô¥£SsU§ÝQdÀGRk‘€˜Ä„ƒWÿÿJP9Z575Z}ÕO ±Fuil€tPIuxÿÿô¥£E…sWÝQd ð ›Çj‘€™ÐI†3UÿÿÿJP9Z5WX9}ÕQ€ ³.Puó€|D Oõr¿ÿô¥£U…§ÝQe éh‡-Wa€–ÖŃYçÿÿ7777 Source: Manual on the GTS, not Manual on Codes

28. 2013-09-11Data Quality Control and Quality Monitoring28 BUFR templates: confusion all around 11 templates for Class 11 reports: Single level report sequences (conventional data), Recommended template: BUFR template for AMDAR, version 7 (ref. tabel 3 11 010, optionally added with 3 11 011 IAGOS template for a single observation, version 2 However all BUFR originating centres produce all different (obsolete) templates. Profile reports are not generated

29. 2013-09-11Data Quality Control and Quality Monitoring29 BUFR templates: confusion all around Source: Manual on Codes

30. 2013-09-11Data Quality Control and Quality Monitoring30 BUFR templates: confusion all around

31. 2013-09-11Data Quality Control and Quality Monitoring31 BUFR templates: confusion all around

32. 2013-09-11Data Quality Control and Quality Monitoring32 