1. Manik Bali Ralph Ferraro , Cheng-Zhi Zou and Lawrence E. Flynn
01 June 2018
Panel Discussion- I In-orbit references for In-Orbit Microwave Instruments
Manik Bali Ralph Ferraro , Cheng-Zhi Zou and Lawrence E. Flynn

2. Outline Introduction GSICS Reference
Existing Methods of Validating Microwave Instruments
Inter-active Discussion
Template
User Survey – Requirements of Microwave Community
Big Question
Proposed Process of Selecting In-orbit Microwave References.
Challenges in search of in-orbit references
Proposed Solution ( Fundamental Climate Data Record)
Application of FCDR as a in-orbit reference ( ATMS-SDR X AMSU-A/MSU FCDR)
Steps for the future.
Summary and Discussion

3. GSICS Introduction
01 June 2018
Global Space Based Inter-calibration System (GSICS) is an international collaborative effort initiated in 2005 by WMO and the CGMS to monitor, improve and harmonize the quality of observations from operational weather and environmental satellites of the Global Observing System (GOS)
Inter- comparisons with reference instrument should aid in re-calibrating monitored instrument -> Reveal trends, temporal . Scan angle and temperature dependent biases.
To meet accuracy and stability requirements of satellite measurements
There is a critical need to perform in-orbit satellite inter-comparison to monitor instrument
Daily inter- comparison of GEO with IASI and AIRS reveal diurnal bias in the GEO (Yu et al 2013)

4. Introduction- Methods for Validating Microwave Instruments
Radiative Transfer Model (RTM)
Model Simulates the Brightness Temperature at the Central Frequency and Polarization of the Channel at the location of the measurement.
Pros: Direct comparison at the point of interest
Cons: Sensitive to quality of biases in input / Uncertainty in SRF
GPS-RO
The GPSRO data are provided by receivers onboard low Earth orbiting (LEO) satellites. As an LEO satellite sets or rises behind the Earth’s limb relative to the GPS transmitter satellite, the onboard GPS receiver takes measurements of the phase and amplitude of the GPS signals.
Pros: Better Accuracy
Cons: Limited Channels/Coverage+ Lower Trop and Mesosphere outside range.
Ideally, absolute bias estimate should be made against a SI-traceable standard of the microwave observations. However, such a standard does not exist .
There is no in-orbit MW instrument that can act as a credible reference capable of detecting full scale of biases and trends through GSICS Style inter-comparisions
Re-Analysis
Reanalysis data are generated from long-term runs of fixed climate modeling systems
with combinations of a comprehensive set of physical parameterizations and
assimilation of routine observations from various sources such as satellites, ships,
buoys, aircraft, and the radiosonde network, etc.
Pros: Global fields of TOA.
Cons: Model Biases and Differences between Re-Analysis datasets.

5. Inter-active Discussion

6. Cheng-Zhi Zou
What are the Instruments and Channels that you wish to monitor or know the bias for
Instruments->  AMSU, ATMS
Channels -> Oxygen Channels
Purpose of monitoring
To Assess instrument performance and issue alerts in case of anomalies
Re-Calibration
Compute biases and provide correction using IMICA approach
Improving downstream services—development of CDR and assessment of CDR being developed
Method of monitoring
GSICS SNO
Inter-satellite difference time series of global ocean mean
What is the Traceability Target
A high quality ( stability ) L1B record developed by IMICA approach

7. Martin Burgdorf
What are the Instruments and Channels that you wish to monitor or know the bias for.
Instruments-> AMSU, MHS, SSMT2
Channels -> Window Channels, 183 GhZ Channels
Purpose of monitoring
Re-Calibration
Compute biases and provide correcMon ( as done for GSICS IR)
Method of monitoring
GSICS SNO
Zonal averages, e. g. over subsidence zones
What is the Traceability Target
One or mulMple In-orbit instruments

8. Moradi- AMSU-B/MHS
What are the Instruments and Channels that you wish to monitor or know the bias for.
Instruments->  MHS, ATMS, SAPHIR, AMSU-B
Channels -> 89GHz and 183 GHz
Purpose of monitoring
To Assess instrument performance and issue alerts in case of anomalies
Re-Calibration
Compute biases and provide correction
Improving downstream services such as WV retrievals, Weather Forecasts etc
Method of monitoring
Using two point intercal technique which includes
Night-time polar averages
Tropical averages
What is the Traceability Target
A high quality ( stability ) L1B record
One or multiple In-orbit instruments

9. Ralph Ferraro (as an end user)
Ralpf Ferraro- As an end User
What are the Instruments and Channels that you wish to monitor or know the bias for.
Instruments->  AMSU, MHS, ATMS, SSMIS, AMSR, GMI, SAPHIR (Initially), but eventually, MWHS2, MWTS2, MWRI (on FY3 series) and MTVZA (on Meteor series).
Channels -> Window Channels, Water Vapor Channels,
Purpose of monitoring
To Assess instrument performance and issue alerts in case of anomalies
Re-Calibration
Compute biases and provide correction ( as done for GSICS IR)
Improving downstream services such as WV retrievals, Weather Forecasts etc
Climate data records
Method of monitoring
Combination of some
Any other
What is the Traceability Target
A high quality ( stability ) L1B record
One or multiple In-orbit instruments – one with a long-term history of stability and minimal orbital drift

10. Wes Berg- GPM/GMI
What are the Instruments and Channels that you wish to monitor or know the bias for.
Instruments->   TMI/GMI
Channels -> Specific Channels
Purpose of monitoring
Method of monitoring
GSICS (SNO & DD)
GPS –RO
NWP
RTM (Model) Vs Instrument
Combination of some

11. Jun Park-KMA- Supporting NWP and CDR on Climate purpose
What are the Instruments and Channels that you wish to monitor or know the bias for.
Instruments->  AMSU, MHS, ATMS, SAPHIR, AIRS, ATOVS, IASI, SSMIs, AMSRs, TMI/GMI, Scatterometers
Channels -> Specific Channels, Oxygen Channels, Window Channels, 183 GHz Channels
Purpose of monitoring
Improving downstream services such as WV retrievals, Weather Forecasts etc
To Assess instrument performance and issue alerts in case of anomalies
Compute biases and provide correction ( as done for GSICS IR)
Method of monitoring
GSICS (SNO & DD)
GPS –RO
NWP
RTM (Model) Vs Instrument
Combination of some

12. Tony - N P R O V S + at STAR 7000 S-NPP +/- 1hr, 50km 4000 IR pass QC
5000
MetOp
+/- 1hr, 50km
2500 IR pass QC

13. Users Survey-Requirements from MW community
Mapping time series of similar sensors but from vastly different heritage (e.g., SSMT2 to AMSU-B) together is of low priority
More precise, longer latency correction are preferred
It does appear most users would look at time series for global trends (most likely the O2 & H2O bands) and use to derive geophysical parameters (most likely window & H2O bands)
The average desired accuracy of the corrections was on the order of 0.4 K (slightly less for the O2 bands)
High quality in-orbit references that can reveal full scale of biases are vital towards meeting these requirements.

14. Big Questions
Can GSICS come up with a procedure to select an in-orbit reference
Can we select a reference to which all MW instruments can trace to.
Can the computed bias help in re-calibrating the instrument so as to permanently remove the biases and trends.
We asked Microwave subgroup members to learn about their requirements
Reference:
Discussions from GSICS Annual Meeting , Delhi India

15. Proposed Selection Process
Selection Process Reference for MW
Identify a instrument and channels one wishes to monitor.
Pre-launch characterization Vs Post launch behaviour reports
Identify the method to monitor
If in-orbit status of key parameters of Candidate Ref instrument are monitored , are within specs and available to users ( via websites such as ICVS)
Consider usage of instruments for similar applications by agencies
OSCAR provides limited but critical information on instruments. (spectral, temporal and geographical coverage)
Detailed information is obtained via ATBD
Consider demonstrated use of the instrument by other agencies

16. Reference Instrument selection Matrix
IR sounder 1. Sensor performance stability 2. Spectral coverage/Spectral resolution 3. Error budgets (prelaunch characterization and post-launch verification) 4. Geo-location accuracy 5. Data availability
References: CrIS, AIRS IASI A/B/C can be GSICS References ( No Primary or Secondary)
References: CrIS, AIRS IASI A/B/C can be GSICS References ( No Primary or Secondary)
Hyper-spectral radiances spanning wide spectrum not available in MW
MW sounder 1. Sensor performance stability 2. Field of view (FOV) consistency (ATMS has oversampling FOV and can be B-G to AMSU-A and MSU) 3. Error budgets (prelaunch characterization and postlaunch verification) 4. Geolocation accuracy 5. Data availability
References: ATMS SDR/TDR, FCDR
Stability Better than -> 1K/dec
Geographic Coverage -> Global or Coverage of Monitored instrument
Spectral Coverage -> Reference instrument should span channels of interest or all channels of a monitored instrument.
Scan Angle dependence of bias needs to be well characterized
User Survey -Expectation from a reference instrument

17. Challenges and limitation in search of In-orbit references
One needs instruments that are several times more stable and accurate than the instruments one wishes to monitor, to serve  as reference. ->Most of the microwave instruments develop biases and trends post launch in their native measurements of Top of Atmosphere Radiances( References Zou and Wang (2011, 2013), issues in prelaunch calibration section).
Microwave channels are broadband, all Microwave instruments do not have exactly the same channels, this means that a limited set of Microwave instruments can be compared directly with each other using GSICS style SNO inter-comparisons that are done in the IR.
After applying GSICS style corrections , the monitored instruments performance can at best trace to the references.
Inter- comparison with a single instrument does not provide a full diurnal variability of bias.

18. Applying Search process in MW
Search began with the target, Monitored Instrument i.e ATMS
First identified channels of ATMS that we wished to monitor.
Aim was to know how well does ATMS –SDR ( L1B BT with Antenna Corrected) perform.
Long run to establish– a Classical GSICS style monitoring of ATMS.
In-orbit instruments such as AMSU and MSU that have exactly same channels as ATMS but would score poorly ( may have known trends scan angle biases nonlinearities ).
Zeroed in on AMSU/MSU FCDR
Trends and biases have been removed
Limb correction similar to Response Vs Scan Angle Correction
Long time series (~ 30 Yrs)
Have highly positive user feedback but not in any known calibration activity

19. AMSU-A FCDR as a Reference for Cross-Calibration
Proposed In-Orbit Reference
AMSU-A FCDR as a Reference for Cross-Calibration
AMSU-A onboard six POES satellites were inter-calibrated using Integrated Microwave Inter-Calibration Approach (IMICA)
5 calibration errors were removed/minimized: nonlinearity, bias drift, frequency shift, sun-heating induced temperature variability in radiances;
Inter-satellite Biases were reduced to K
19 years of swath data
Dataset available from NCEI CDR website
AMSU-A brightness temperature s for overlapping NOAA-15 and
NOAA Their differences for randomly selected region
(e.g., within the dashed square ) are within K.
Series of MSU/AMSU-A provide a long term reference records spanning over a large range of time and full span of scan angles.

20. AMSU-A FCDR of Window Channels
Proposed In-Orbit Reference
AMSU-A FCDR of Window Channels
AMSU-A onboard six POES satellites were inter-calibrated using Integrated Microwave Inter-Calibration Approach (IMICA) for window channels, same as sounding channels
Scan bias, nonlinearity and bias drift errors were removed/minimized;
Inter-satellite standard deviation improved by 50%, comparing to 1b data sets;
Swath data from 1998 to present
Dataset available from NCEI CDR website
As accompaniments to AMSU-A FCDR of sounding channels, the window channels cover channels 1, 2, 3 and 15 (23.8, 31.4, 50.3 and 89.0 GHz), which are widely used in many other sensors, such as SSMI, SSMIS, ATMS, GMI and etc. These channels are mainly for the input to hydrological products, and land surface products.

21. Overlap with Monitored instruments
AMSU/MSU
FCDR
AMSR (2/E)
JAXA
MWTS-2
China
SSMI
DOD
MTVZA
ROSCOSMOS/Ukraine
GMI
NASA
ATMS
MSU/AMSU-A FCDR channels overlap with majority of MW instruments
Source:

22. Stability of FCDR AMSU-A/MSU FCDR continuously monitored in Real time
Scan Angle Dependence has been corrected
Validated with GPSRO and instruments monitored at ICVS

23. Monitoring ATMS-SDR by SNO inter- comparison with N18
Inter- comparison of ATMS-SDR with FCDR show
Low scan angle dependence of ATMS-SDR
Post launch ATMS-SDR maintains nearly pre-launch level of accuracy.

24. Monitoring ATMS-SDR by SNO inter- comparison with FCDR Aqua
FCDR extremely consistent as a reference  Compared to N18, Inter- comparison with AQUA-AMSU FCDR gives similar results

25. Steps for the Future
Do we need climate accurate records for MW instrument monitoring?
Can we extend to Window and 183 GHz channels ( Algorithms Available)
How do we monitor channels for which we don’t have FCDR.
Open Challenges
Differ ring scanning geometry of inter-comparing instruments.
Differing Central Frequency of inter-comparing instruments.
What is should be the format of the FCDR ( For eg a typical scene that is climate accurate which is similar to the AMSUA-A FCDR)

26. Summary
Rules of identifying in-orbit reference for Microwave instruments have been suggested.
Discussions within the community have suggested a wide range of Microwave instruments that need to be monitored using IMICA techniques and Vicarious techniques ( such as the Moon).
It is proposed that AMSU-A/MSU FCDR can be used as a high quality in-orbit reference for monitoring MW Channels of In-orbit instruments.
The AMSU-A/MSU FCDR has nearly pre-launch level of stability and can reveal wide range of
It is suggested to use IMICA technique to produce FCDR records for Window channels and 183 GhZ channels to provide a record spanning larger group of Microwave instruments.

27. THANK YOU

28. Template
What are the Instruments and Channels that you wish to monitor or know the bias for.
Purpose of monitoring ( Re-Calibration, Correcting the instrument for biases, Improving downstream services such as Forecasts etc)
Method of monitoring
GSICS SNO
GPS –RO
RTM (Model) Vs Instrument
Combination of all
Traceability Target ( 1 Target to which all instruments to be traceable or different).

29. M. Burgdorf’s Answers
What are the Instruments and Channels that you wish to monitor or know the bias for.
Instruments-> AMSU, MHS, SSMT2
Channels -> Window Channels, 183 GhZ Channels
Purpose of monitoring
Re-CalibraMon
Compute biases and provide correcMon ( as done for GSICS IR)
Method of monitoring
GSICS SNO
Zonal averages, e. g. over subsidence zones
What is the Traceability Target
One or mulMple In-orbit instruments

30. Summary and Conclusion
A process of selecting reference instrument has been proposed that takes into account subgroup needs, User expectation and Scoring.
Search could begin with first identifying the instrument and the channels one wishes to monitor and then go on to identify the reference candidate/s
In a multiple reference instruments scenario , instruments complement each other and not compete with each other-> Propose to use ‘GSICS Reference’ and not ‘Primary/Secondary GSICS Reference’.
EUMETSAT proposed scoring scheme comprehensive and can be simplified and applied to specific situations and needs.
Stability of Reference instrument bias and health, Scan Angle bias characterization, Overlap of Spectrum with Target instrument etc can be simple criterion for selection
Proposed Reference instruments
For IR- IASI-A/B/C/CrIS/AIRS
For VIS - Aqua-MODIS ( Pre- 2012) and VIIRS( Post- 2012)
For MW- ATMS-SDR/TDR , FCDR.

31. Proposed VIS/NIR Instrument References
The reference sensor weighting scheme is then the overall uncertainty to transfer the SI traceable calibration reference to the target reference
Pre 2012 Aqua-MODIS
Post 2012 use VIIRS, VIIRS onboard calibration superior and better characterized than MODIS
Evaluate the 3rd generation GEO and Metop-SG calibration stability and reconsider VIS/NIR reference instrument
Also worth to have several reference records established
To monitor the primary reference
In case of instrument failure and lack of overlap

32. Thank You

33. Introduction- Instrument resources
OSCAR [ Observing System Capability Anaysis and Review Tool]
OSCAR is a resource developed by WMO in support of Earth Observation applications, studies
and global coordination. (IASI , CrIS )
Web pages are static. ( can act as a starting point but not complete input for selecting Ref Instrument)
National Calibration Center (NCC)
Info on NCC instrument that is more relevant to Cal/Val community
ICVS [ Integrated Calibration Validation System ] ( other similar systems can be explored)
On-orbit and prelaunch instrument characterization and long-term monitoring of instrument performance
Web pages give dynamic info on key cal/val parameters of the instrument

34. Introduction-Selection of GSICS References for Classical Monitoring
Initial years of GSICS ….handled problems of lower complexity ….
Monitored broad band GEO /LEO sensors by inter comparing with hyper spectral LEO in morning evening and afternoon.
Needed a reference instrument that is stable enough be able to monitor GEO/LEO instruments and reveal expected measurements biases ( and variations in them) .
AVHRR
The MODIS instrument has high calibration accuracy and can track its own radiometric changes due to the inclusion of an onboard calibration system for the VIS/NIR spectral region [MCST].
Stated Accuracy better than 1 K.
AIRS ( Afternoon orbit) and IASI ( Morning orbit ) Hyper-spectral Instruments
Radiances span main IR channels of broad band instruments i.e Micron
AIRS and IASI routinely monitored and long term monitoring of three way comparisons exist ( CrIS IASI and AIRS).
Added confidence Comparisons of IASI and AIRS with Climate Instruments (ATSR2 and AATSR) reveal better than 0.1 K accuracy and pre-launch like behavior of AIRS and IASI.
In-orbit reference Instrument noise not fully known
Used a simple selection criterion for selecting reference instrument
Focused on monitoring IR and VIS bands using a limited number of reference instruments
MODIS
IASI
AIRS
For In-orbit Inter-calibration we broadly needed
An algorithm that can perform the inter- comparison [ eg. DCC, SNO baseline algorithm]
Well calibrated in-orbit References [ stability, accuracy ]
MTSAT
SEVIRI
GOES

35. Current State of In-Orbit Monitoring
01 June 2018
GRWG IR MW UV
VIS
AIRS
IASI A/B/C
Primary Ref*
CrIS*
GP-X *
ATMS*
MSU AMSU
SSMI*
OMI*
Aqua Modi+ DCC
Complexity of in-orbit Inter - comparision enhanced as by surface reflectance Solar Zenith Angle. viewing geometry impact A-B .
Stability of Transfer targets such as DCC, Desert kicks in instrument monitoring algo.
MW spectrum is large and not spanned by a single reference instrument.
Multiple broad band instruments can be candidates.
Monitored instrument overlaps with spectrum of hyper-spectral instruments
tore
Diverse requirements across subgroups ( even within subgroups )

36. Opportunities and Challenges
Multiple candidate reference instruments ( CrIS IASI and AIRS) provide opportunity to monitor GEO instruments bias diurnally and give a complete picture of the bias variation.
Inter- comparison between the multiple instruments when considered along with their health monitoring can provide confidence about thrust worthiness of the instruments.
Gives an opportunity for inter-operability of computed bias among reference instruments, gives a challenge to develop advanced algorithms to achieve a trustworthy inter-operability.

38. Tim Proposed Scoring Scheme-Overview
A very simple scheme
The proposed scoring scheme can be summarized by the definition of the following for each class of GSICS product:
Variables describing required performance of Reference.
Minimum thresholds for each variable required for inter-calibration to be useful.
Maximum thresholds (optimal values) for each variable above which no further improvement will be realized
Weighting for each variable (to align with contributions to overall uncertainty)
Curtsey Tim Hewison

39. Application of Process- Example-1
Selection Process Reference for MW
A sub-group meeting is organized to identify instrument to be monitored.
Instrument info on OSCAR and NRT monitoring on ICVS is considered
OSCAR provides limited but critical information on instruments. (spectral, temporal and geographical coverage)
Detailed information is obtained via ATBD
Group wishes to monitor GPM satellites spread over a range of roughly 25S to 25N ( for eg GMI)
Despite not presenting global observations SAPHIR also scores higher because the intended goal is to inter-calibrate low inclination instruments. In addition input from scoring is considered.
Both SAPHIR and ATMS can score the same marks
Group evaluates ATMS and SAPHIR.
On the other hand if goal would have been to monitor Polar instruments ATMS could have been a better choice.
Group considers that SAPHIR are the first radiometers in the 183 GhZ in low inclination orbit ( Wilheit)

40. Application of Selection Process - Example-2
Reference for IR
A sub-group meeting is organized to identify instrument to be monitored.
Group considers that both the instruments span the Monitored instrument channels
Group wishes to monitor 11 and 12 and 13 Micron Channels of a GEO instrument.
OSCAR page reveals that noise levels in the Micron range are comparable.
Monitoring at ICVS reveals key calibration parameters within spec .
Group evaluates IASI and CrIS
Both instruments can act as references .

41. Summary and Conclusions
Process by which reference instrument could be selected in GSICS has been proposed here.
It is proposed that the main decision to identify a reference instrument should rest with the subgroup.
Evaluation of OSCAR for the purpose of Ref selection reveals that while some information about candidate reference instruments could be obtained via the WMO OSCAR however OSCAR pages are static and a complete info about instrument should be obtained from the agency maintaining the instrument. Special caution to be applied when considering noise levels ( NeDT) stated at OSCAR.
More useful input could be provided by instrument monitoring done by the agency ( such as ICVS, NCC).
Scoring scheme for reference instrument can be a vital input in deciding which instrument to use as a reference.