1. Fiduceo update GRWG meeting 2019
Frank Rüthrich, Viju John, and Rob Roebeling (EUMETSAT)
Rhona Phipps and Chris Merchant (University of Reading)

2. Fundamental Climate Data Records Wrap-Up
Content
Fiduceo introduction
Principles metrology
Fundamental Climate Data Records
Wrap-Up
RP:
FIDUCEO – reminder of aims/ objectives ( as perhaps some people will have already heard some things about the project)
FIDUCEO metrology
Putting this metrology into action - FIDUCEO FCDRs to CDRs
2019 final activities

3. Fiduceo Introduction

4. Fiduceo – Key Idea comprehensive and realistic;
FIDUCEO aims to develop and apply a rigorous metrology* of Earth observation for climate data records
The project is focused on the uncertainty budget
comprehensive and realistic;
communicated using internationally agreed standards;
traceable from detector to gridded geophysical product; Traceability here we talk about the property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty
Note reference does not have to be SI, only community agreed;
propagated rigorously across product levels.
Using uncertainty analysis the project will provide Fundamental Climate Data Records (FCDRs) and Thematic Climate Data Records (TCDRs) with pixel level uncertainties.
RP:
FIDUCEO – reminder of aims/ objectives
FIDUCEO aims to develop and apply a rigorous metrology* of Earth observation for climate data records
The project is focused on better characterisation of the uncertainties
comprehensive and realistic;
communicated using internationally agreed standards;
traceable from detector to (in some cases) a gridded geophysical product; Traceability here we talk about the property of a measurement result whereby the result can be related to a reference through a documented unbroken chain of calibrations, each contributing to the measurement uncertainty
Note reference does not have to be SI, only community agreed;
propagated rigorously across product levels.
Using the uncertainty analysis, the project will provide Fundamental Climate Data Records (FCDRs) with pixel level uncertainties.
The project team will then show how these FCDRs and their uncertainties can be used to produce Thematic Climate Data Records (TCDRs) with propagated traceable uncertainties.
*Metrology is the science of measurement and measurement uncertainty

5. Fiduceo FCDRs Recalibrated
Next 2 slides just shows the main FCDRS, CDRs to be developed.
Bear in mind that there will be also the documentation/ tools/ cook books to go with them..
FCDRs of AVHRR and HIRS for AATSR-to-SLSTR gap period – Month 36 (Feb 2018) – not on timeline slide, depends on availability of the SLSTR data
RP: Needs amending – MW – not harmonised now,
Optional - FCDRs of AVHRR and HIRS for AATSR-to-SLSTR gap period

6. Fiduceo CDRs Microwave sounders
RP: needs updating – UTH only from MW at the moment.

7. Principles Metrology

8. Uncertainty from low-level data propagates to high-level information
Principles metrology
Raw satellite data (L0)
Calibrated radiances (L1)
Climate data record (L2)
Gridded CDR (L3)
Analysed / processed (L4+)
Climate index / information
Decision
Insurance
Liability
How sure are we of the information? - Uncertainty
How defensible is the information? - Traceability
Each transformation injects new uncertainty
Uncertainty from low-level data propagates
to high-level information

9. Principles metrology FIDUCEO addresses fundamental
gaps in capability that prevent
EO-based CDRs and climate information having traceable uncertainty.
Raw satellite data (L0)
Calibrated radiances (L1)
Climate data record (L2)
Gridded CDR (L3)
Analysed / processed (L4+)
Climate index / information
The FIDUCEO techniques could be applied to current and future C3S programmes.
Status
FCDRs – Q2 2019
CDRs – Q2 2019
Metrology cookbooks and “how to” guides – available now.
Workshop 1 completed 2018.
Workshop 2 – Q2 2019
Decision
Insurance
Liability

10. Example uncertainty tree

11. MVIRI FCDR Rob Roebeling, Frank Ruethrich, Viju John, Yves Govearts, Ralf Quast, Ralf Giering

12. FCDR MVIRI VIS
Spectral response functions of Meteosat-2,-3,-4,-5,-6,-7 were recovered and reconstructed;
FCDR has been generated and is available in NetCDF format;
FCDR includes uncertainty information that is provided at pixel level. Uncertainties provided are independent, structured, and common uncertainties (see definitions at
FCDR was evaluated for time series over known targets as well as through comparison against SCIAMACH Y observations.

13. FCDR MVIRI VIS - Reconstruction Spectral Response Function -
Simulated count
comparison
Measured count
Convolution with (prior) SRF
Optimize SRF
WP partners:
Courtesy: Rayference

14. FCDR MVIRI VIS - Simulation and reconstruction SRFs -
1.0
0.8
0.4
0.6
0.0
0.2
Responsivity (max = unity]
Wavelength (mm)
METEOSAT-2 reconstructed SRF
1.0
0.8
0.4
0.6
0.0
0.2
Responsivity (max = unity]
Wavelength (mm)
METEOSAT-3 reconstructed SRF
1.0
0.8
0.4
0.6
0.0
0.2
Responsivity (max = unity]
Wavelength (mm)
METEOSAT-4 reconstructed SRF
1.0
0.8
0.4
0.6
0.0
0.2
Responsivity (max = unity]
Wavelength (mm)
METEOSAT-5 reconstructed SRF
1.0
0.8
0.4
0.6
0.0
0.2
Responsivity (max = unity]
Wavelength (mm)
METEOSAT-6 reconstructed SRF
1.0
0.8
0.4
0.6
0.0
0.2
Responsivity (max = unity]
Wavelength (mm)
METEOSAT-7 reconstructed SRF

15. FCDR MVIRI VIS - Harmonised Timeseries -
+ Met 2
+ Met 3
+ Met 4
+ Met 5
+ Met 6
+ Met 7
Figure: Time-series of EUMETSAT (MFG) harmonised VIS FCDR over an Algerian, Nile, and Ocean target

16. FCDR MVIRI VIS - Homogenised Timeseries -
+ Met 2
+ Met 3
+ Met 4
+ Met 5
+ Met 6
+ Met 7
Figure: Time-series of EUMETSAT (MFG) homogenised VIS FCDR over an Algerian, Nile, and Ocean target

17. FCDR MVIRI VIS - Simultaneous Nadir Overpass comparison against SCIAMACHY -
Target: Atlantic Ocean
Target: Keynia
Figure: Comparison of MVIRI visible channel radiances against SCIAMACHY simumated MVIRI radiances for Simultaneous Nadir Overpasses over the Atlantic Ocean and Keynia.

18. AVHRR FCDr Jon Mittaz, Mike Taylor

19. FCDR AVHRR IR
An unharmonized pre-beta FCDR of AVHRR radiances has been generated from NOAA-6 till NOAA-19 and from Metop-A and Metop-B covering 1979 – 2017;
Examples of improvements made to the pre-beta AVHRR FCDR include:
better correction of Internal Calibration Target temperatures,
better filtering of outliers,
variable noise estimates,
better understanding of error correlations.
The pre-beta AVHRR FCDR includes uncertainty information at pixel level, as well as error correlation matrices. Uncertainties provided as independent, structured, and common uncertainties (see
The work to harmonize the AVHRR FCDR is still ongoing. Availability of representative satellite to satellite match-ups is hindering robust harmonisation;
The aim is to provide a harmonized version of the AVHRR FCDR by the end of Fiduceo (in September 2019) in NetCDF format.

20. FCDR AVHRRs - Internal Calibration Target (ICT) temperature correction -
Comparison of PRT temperatures on the ICT for NOAA-12 and NOAA-17. NOAA-12 shows large deviations, indicative of larger thermal gradients across the ICT
T4.4: Statement of the problem: Error on the ICT temperature estimates due to thermal gradients
T4.4: Objective: to account and correct for changes in the thermal gradients across the ICT. These, if not corrected, can introduce larger errors in the estimate of the ICT radiant temperature
ICT Temperature correction
A model of the ICT temperature error is fitted to force the 3.7 mm channel gain to be as close to a constant as possible (the InSb detector is expected to have a linear response for this channel)
Uncertainty estimated from residual variation of 3.7 mm channel gain
ICT temperature corrected temperature also used in the calibration of the 11 and 12 mm channels.
LH plot shows the 3.7 micron channel gain for NOAA-12 before correction (blue) and after correction (orange). RH plot shows the change in the ICT temperature values
PRT: Platinum Resistance Thermometers
ICT: Internal Calibration Target

21. HiRs FCDr Gerrit Holl, Jon Mittaz, Viju John

22. FCDR HIRS
An unharmonized pre-beta FCDR of HIRS radiances has been generated from NOAA-6 till NOAA-19 and from Metop-A and Metop-B covering 1979 – 2017;
Examples of improvements made to the pre-beta HIRS FCDR include:
new self-emission model,
variable noise estimates,
better understanding of error correlations.
The pre-beta FCDR includes uncertainty information at pixel level, as well as error correlation matrices. Uncertainties provided as independent, structured, and common uncertainties (see
The work to harmonize the FCDR is still ongoing. Availability of representative satellite to satellite match-ups is hindering robust harmonisation;
The aim is to provide a harmonized version of the HIRS FCDR by the end of Fiduceo (in September 2019) in NetCDF format.

23. Task 4.3 – FCDR HIRSs
Figure: EasyFCDR output of HIRS orbit with Brightness Temperatures (BT), independent uncertainty, and structured uncertainty for channel 4 (14.2 µm)

24. FCDR HIRS
Self emission issues?
Figure: Time-series of HIRS channel 12 (6.5 or 6.2 µm) Brightness Temperatures before harmonisation (upper panel) and after harmonisation (lower panel)

25. MW FCDr Martin Burgdorf, Imke Hans

26. FCDR Microwave Humidity Sounders - MHS, SSM/T2, AMSU-B -
FCDR is generated and available in NetCDF format;
Scientific improvements made to the FCDR are: i) Correct natural constants, ii) Correct identification of Moon in DSV, iii) 6 Platina Reference Targets included, iv) different band corrections for internal warm calibration target and deep space view, v) Appropriate antenna pattern correction, vi) RFI correction applied to all satellites
FCDR includes uncertainty information that is provided at pixel level. Uncertainties provided are independent, structured, and common uncertainties (see definitions at

27. FCDR Microwave Humidity Sounders - Comparison of the timeseries of TB in Channel 3 -
The FCDR from FIDUCEO provides more consistent and stable time series
The shaded regions denote the uncertainty due to common effects for FIDUCEO
There is no uncertainty estimate on pixel level for the products of AAPP

28. 2nd FIDUCEO WORKSHOP on CDRs

29. To register for the 2nd Fiduceo workshop please check the website:
FIDUCEO 2nd Workshop
Topic: New climate Data from space: Exploitation for Science and Services
Date: 25th to 27th June 2019
Location: IPMA Alges – Lisbon, Portugal
Summary: Workshop comprises of a dedicated training session followed by 4 workshop sessions:
Fundamental Climate Data records (FCDRs) principles and methods.
Deriving Climate Data Records (CDRs) from FCDRS
CDR Quality assurance
CDR Exploitation
To register for the 2nd Fiduceo workshop please check the website:

30. Wrap-up

31. Fiduceo 5 March 2019
Final versions MVIRI (harmonised) and MHS (reprocessed) FCDRs available;
Pre-beta versions HIRS (reprocessed) and AVHRR (reprocessed) FCDR available;
Pre-beta and final versions of FCDRs provided to early users for testing;
Completed preparing matchups needed for harmonisation process;
Harmonisation procedure is being tested for HIRS and AVHRR.
Algorithm Theoretical Baseline Description and Product User Guides to FCDRs ready in draft;
Finalised plans and advertised the second workshop (June 2019);
Published or submitted several papers on the Fiduceo FCDR work.
(Could also add – outreach to external agencies working well – ESA workshop on Uncertainties was well received and commented on by project officer and reviewer)
RP:
FIDUCEO Metrology process:
Defined in the deliverables D2_2 (for FCDRs) and D2_4 (for CDRs).
Tutorials on the website
FIDUCEO metrology paper currently in review process
FCDRs:
MVIRI FCDR ready to be released.
MW FCDR ready to be released (uncertainty characterised only)
AVHRR/ HIRS harmonised FCDRs still in preparation.
pre-beta versions of some of the FCDRs have been released to early users for testing.
CDRs:
Aerosol/Albedo CDR in processing stages
Aerosol CDR (from AVHRR) in preparation
UTH CDR close to final status (derived from MW FCDR)
SST ensemble in early preparation work
Documentation/ writeups in progress – technical documents, Product User Guides, release notes, Quick start guides.
Continued working on finalising the format/content of the FCDRs and CDRs.
Matchup system working well.(needed as input to harmonisation process)
harmonisation process still in progress.
Outreach
Held first workshop (April 2018) – good feedback.
Registration is open for the second workshop (June 2019). Preparation is underway.

32. Fiduceo open tasks till September 2019
To complete FCDRs (AVHRR, HIRS, MW)/ Beta (MVIRI) and deliver to beta users to test;
To consolidate CDR uncertainty analysis across the CDR teams;
To consolidate new CDR format;
To create CDRs and deliver to beta users to test;
To present FIDUCEO analysis at Workshop 2 and gather feedback from beta users;
To write up produce user guides and cookbooks;
To ensure continuation of project interaction with external agencies.
Still a lot to do.
3 short bullet points – doesn’t reflect the substantial amount of effort involved here!!
Consolidate CDR uncertainty analysis across the CDR teams
Consolidate new CDR format
Create CDRs and issue to beta users
Ensure continuation of project interaction with external agencies. (like GSICS….. ;-) )
RP:
To complete FCDRs (AVHRR, HIRS, MW)/ Beta (MVIRI) and deliver to beta users to test
To consolidate new CDR format
To create CDRs and issue to beta users
To present FIDUCEO analysis at Workshop 2 and the feedback from beta users
To finalise user guides and tutorials/ cookbooks
To ensure continuation of project interaction with external agencies.

33. Involvement in international scientific fora
Presentations at scientific conferences
Burgdorf, Martin, Stefan Buehler, Imke Hans, Theresa Lang, Simon Michel, 2016: The Moon as Possible Calibration Reference for Microwave Radiometers, EGU, April 2016.
Govaerts, Y., R. Quast, F. Ruethrich, R. Giering, R. Roebeling, 2016, Recovery of MVIRI/VIS band spectral response, the 2016 EUMETSAT Meteorological Satellite Conference, September 2016, Darmstadt, Germany.
Holl, G., J. Mittaz, and C. J. Merchant, 2017: A new Fundamental Climate Data Record (FCDR) for almost 40 years of measurements from the polar-orbiting High resolution Infrared Radiation Sounder (HIRS) based on a metrologically traceable uncertainty analysis EMS Annual Meeting, 4-8 September, Dublin, Italy
Ruethrich, Frank, Ralf Quast, Yves Govaerts, Viju O. John, Ralf Giering, Rob Roebeling, Sebastien Wagner, Tim Hewison, and Joerg Schulz, 2017: Relevant Sources of Uncertainty in a Climate Data Record from the Meteosat Visible Channel, 2017 EMS Annual Meeting, 4-8 September, Dublin, Italy
Burgdorf, Martin, Theresa Lang, Marc Prange, Stefan A. Buehler, and Imke Hans, 2017: In-Flight Characterization of Microwave Sounders With the Moon, the 2017 EUMETSAT Meteorological Satellite Conference, 2-6 October 2017, Rome, Italy.
Hans, Imke; Burgdorf, Martin; John, Viju O.; Mittaz, Jonathan; Buehler, Stefan A., 2017: Noise performance of microwave humidity sounders over their lifetime, the 2017 EUMETSAT Meteorological Satellite Conference, 2-6 October 2017, Rome, Italy.
Govaerts, Y., R. Quast, F. Ruethrich, R. Giering, M. Luffarelli, V. John, J. Schulz, 2017: Generation of aerosol and surface reflectance Climate Data Record from MVIRI/VIS observation: impact of the reverse engineered sensor spectral response on Level-2 product retrieval, the 2017 EUMETSAT Meteorological Satellite Conference, 2-6 October 2017, Rome, Italy.
Holl, G., J. Mittaz, and C. J. Merchant, 2017: A new Fundamental Climate Data Record (FCDR) for almost 40 years of measurements from the High resolution Infrared Radiation Sounder (HIRS) based on a metrologically traceable uncertainty analysis EUMETSAT Meteorological Satellite Conference, 2-6 October 2017, Rome, Italy.
Ruethrich, Frank, Ralf Quast, Yves Govaerts, Viju John, Ralf Giering, Rob Roebeling, Sebastien Wagner, Tim Hewison and Joerg Schulz, 2017: Impact of reverse engineered MVIRI visible channel SRFs on reflectance FCDR, the 2017 EUMETSAT Meteorological Satellite Conference, 2-6 October 2017, Rome, Italy.
Quast, Ralf; Govarts, Yves; Rüthrich, Frank; Giering, Ralf, 2017: A novel method for vicarious re-characterisation of the MVIRI VIS spectral response to facilitate climate monitoring. EMS Annual Meeting 2017, Dublin, 4-8 September
Quast, Ralf; Govaerts, Yves; Rüthrich, Frank; Giering, Ralf, 2017: A novel method for vicarious re-characterisation of the MVIRI VIS spectral response to facilitate climate monitoring. EUMETSAT Meteorological Satellite Conference, Rome, 2-6 October
Hanschmann, Timo, Gerrit Holl, Viju John, Rob Roebeling, Jörg Schulz, 2017: Recalculated HIRS radiances as input to Reanalysis within the Copernicus Climate Change Service, ICR5, November 2017
Hanschmann, Timo, Gerrit Holl, Viju John, Rob Roebeling, Jörg Schulz, 2017: Preparing HIRS radiances as input to Reanalysis within the Copernicus Climate Change Service, ITSC-XXI: Darmstadt, 29 November - 5 December 2017
Holl, G., J. Mittaz, and C. J. Merchant, 2017: A new Fundamental Climate Data Record (FCDR) for nearly 40 years of measurements from the High resolution Infrared Radiation Sounder (HIRS) based on a metrologically traceable uncertainty analysis. ITSC-XXI: Darmstadt, 29 November - 5 December 2017
Prange, Marc et al., 2017: Investigation and an approach for a vicarious definition of natural calibration targets for satellite based microwave sensors, ITSC-XXI: Darmstadt, 29 November – 5 December 2017
M. J. Burgdorf, I. Hans, and S. A. Buehler, 2018: New (Fundamental) Climate Data Records From Microwave Sounders With FIDUCEO, 11. Deutsche Klimatagung, 5-8 March 2018, Frankfurt, Germany.
Note: pre-versions may be available earlier for test reanalysis. So watch out on the website under data access/climate. 33

34. Involvement in international scientific fora
Conference proceedings
Govaerts, Y., R. Quast, F. Ruethrich, R. Giering, R. Roebeling, 2016: Recovery of MVIRI/VIS band spectral response. In Proceedings of the 2016 EUMETSAT Meteorological Satellite Conference, 2-6 September 2016, Darmstadt, Germany.
Rüthrich, Frank, Viju O. John, Rob Roebeling, Sebastien Wagner, Bartolomeo Viticchie, Tim Hewison, Yves Govaerts, Ralf Quast, Ralf Giering, Jörg Schulz, 2016: A Fundamental Climate Data Record that accounts for Meteosat First Generation Visible Band Spectral Response Issues. In Proceedings of the 2016 EUMETSAT Meteorological Satellite Conference, 2-6 September 2016, Darmstadt, German
Peer reviewed papers
Burgdorf, M. J., S. A. Buehler, T. Lang, S. Michel, I. Hans, 2016: The Moon as a photometric calibration standard for microwave sensors, Atmospheric Measurement Techniques , 9,
Hans, I., Burgdorf, M., John, V. O., Mittaz, J., and Buehler, S. A.: Noise performance of microwave humidity sounders over their lifetime, Atmos. Meas. Tech., 10, , , 2017
Burgdorf, M., Hans, I., Prange, M., Lang, T., Buehler, S.A., 2018, Inter-channel uniformity of a microwave sounder in space, Atmospheric Measurement Techniques 11, 4005–4014.
Hans, I., Martin Burgdorf, Stefan A. Buehler, Theresa Lang, Marc Prange, Viju O. John: 2019: An uncertainty quantified fundamental climate data record for microwave humidity sounders, Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records”(accepted)
Giering, Ralf, Ralf Quast, Samuel Hunt, Peter Harris, Jonathan Mittaz, Emma Woolliams, 2019: Harmonisation of satellite sensors, Remote Sensing (submitted)
Govaerts, Yves, Frank Rüthrich, Viju John, and Ralf Quast, 2018: Generation of accurate top-of-atmosphere spectral radiance over pseudo-invariant calibration sites for the recovery of MVIRI/VIS band spectral response, Remote Sensing (accepted)
Holl, Gerrit, Jonathan P. D. Mittaz, Christopher J. Merchant, 2018: Error covariances in High-resolution Infrared Radiation Sounder (HIRS) radiances, Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records“(submitted)
Merchant, Christopher J.,Gerrit Holl, Jonathan P.D. Mittaz, Emma R. Woolliams, 2018: Radiance uncertainty characterisation to facilitate Climate Data Record creation, Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records“(accepted)
Quast, Ralf, Ralf Giering, Yves Govaerts, Frank Rüthrich, and Rob Roebeling, 2019: Climate Data Records from Meteosat First Generation Part II: Retrieval of the In-Flight Visible Spectral Response, Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records“ (accepted)
Tabata, V.O. John, R.A. Roebeling 2, T. Hewison and J. Schulz, 2019: Re-calibration of over 35 years of infrared and water vapor channel radiances from the JMA geostationary satellites, Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records“ (submitted)
Viju O. John, R. A. Roebeling, T. Tabata, T. Hewison, and J. Schulz, 2019: On the recalibration of the infrared (IR) and water vapour (WV) radiance time-series from European and Japanese geo-stationary satellites , Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records“ (submitted)
Frank Rüthrich, Ralf Quast, Yves Govaerts, Viju O. John, Rob Roebeling, Emma Wooliams and Joerg Schulz, 2019:Climate Data Records from Meteosat First Generation Part III: Recalibration and Uncertainty Tracing of the Visible channel on Meteosat-2-7 using Reconstructed, Spectrally Changing Response Functions , Remote Sensing Special Issue "Assessment of Quality and Usability of Climate Data Records“ (submitted)
Note: pre-versions may be available earlier for test reanalysis. So watch out on the website under data access/climate. 34

35. Thank You