1. Strategy for combining corrections for VIS/NIR+IR channels and plotting tool
S. Wagner, T. Hewison (EUM)
M. Takahashi (JMA)

2. Product generation: current approach
Topical/spectral separation:
VIS/NIR IR
Choice of the reference instrument(s):
IASI-A, IASI-B, HIRS.
MODIS, VIIRS
How does it look like on GPRCs GSICS servers?
Many areas (instruments, demo, pre-op, ope) with many products
Simplified thanks to the prime calibration products
Current catalog structure could support one unique product
What about injecting the GSICS products into Level 1 data?
CGMS recommendation
Similar to having one merged product

3. Product specificities… from high level…
IR products:
Daily product based on collocations with the reference instrument
One unique method ⇒ one unique algorithm ⇒ one evolution through the distribution mode (demo, pre-op, ope)
Versioning is straightforward
VNIR products:
Multiple methods + a blend ⇒ more algorithms
Potentially, different time line to derive the products from each individual methods (i.e. DCC and Lunar products)
Evolution through the product acceptance still to be discussed:
Each method evolves separately: DCC, Moon, Rayleigh, Blend
What about the combined product?
What about the versioning (individual methods, blend, changes in the IR product [the DCC uses the IR])?

4. Product specificities… to the details…
File naming:
No unique reference instrument anymore  addressed by the PRIME products
Product can be only identified by its version number
Global attributes  Some attributes are specific to the spectral range and/or the methods. For example:
Geospatial_lat_min / geospatial_lat_max
Geospatial_lon_min / geospatial_lon_max
Reference_instrument
Reference_instrument_wmo_code
Etc.
⇒ Update of some global attributes to reflect the change of contents (title, history, summary, keywords, etc.)
⇒ Some attributes may need to move from global attributes to variables in order to account for more values depending on the spectral range / method

5. Product traceability
How to address product history and processors version?
⇒ Software traceability (in particular in the case of the VIS/NIR for which more algorithms are in place)
⇒ Need to agree on the process for product development cycle to ensure coherence across GPRC.
⇒ Can SCOPE-CM experience help?
For instance: MSG1 IR OPE product:
:history = “ Collocations done by IasiSeviriCollocator.java  revision 140 ;\n“, “ NetCDFiles generated by gsics.rep.pl revision 86;”
What about the window period? Is it the same for all methods (VIS/NIR and IR)?
What about the DOI?
Different processing level may be conflicting with the assignment of a DOI.
Can we blend DOIs? (question by Masaya)

6. Traceability: the example of the EUM lunar calibration system

7. Product specificities… back to the details…
Dimensions:
Only one needs to be added ( = number of methods for the VIS/NIR).
Do we want to consider the SNOs for the IR as a method (weight = 0 for the VIS/NIR)?
Variables
Variable
Scope
Comment
date
All channels
No change
channel_name
No change (in the VIS/NIR, the platform’s name needs to be added ⇒ opportunity to move to Meteosat instead of MSG?)
central_wavelength
wnc
IR (+ VIS/NIR)
Extendable to all channels.
number_of_collocations IR
Not extendable to VIS/NIR
std_scene_tb
mon_slope / slope
Ambiguity: for IR, is this an absolute slope or the slope of the correction? In the VIS/NIR, it is the slope of the absolute gain ⇒ The scope of this variable needs to be clarified + units to be updated + attributes to be updated to reflect the change.
mon_offset / offset
Problem with the units between IR and VIS/NIR + meaning of the offset to be clarified

8. Product specificities… back to the details…
Variable
Scope
Comment
slope_se
All channels
Not provided yet for VIS/NIR + problem with units between VIS/NIR and IR
offset_se
covariance
Std_scene_tb_bias / std_scene_tb_bias_se IR
Not extendable to VIS/NIR
alert
All channels?
Could be extended to VIS/NIR. Should be specified for each channel.
validity_period
Potentially specific to each method, but in practice could be the smallest validity period
Alpha / Beta
method_name
Extendable to all channels
weight_method
For the IR only one method currently (SNOs) ⇒ matrix full of 0 if the current approach for the VIS/NIR is extended to the IR channels
Mon_official_slope / mon_official_offset
For the moment only provided in the VIS/NIR product. To be discussed

9. Feedback from Masaya
Remove method/spectral specific variables/attributes as much as possible.
⇒ All the variables are expected to have dimensions of # of channels (e.g. 12 for SEVIRI). If variables cannot be extendable to all channels, we could fill those channels’ values with _FillValue (missing value).
One variable can contain one units – could be an issue for variables such as inter-calibration coefficients. E.g., GEO-LEO-IR adopts radiance in wavenumber, but JMA distributes L1 radiance in wavelength for all VNIR/IR.
⇒ Many users of JMA GEO imagers (e.g. AHI) need to convert radiance units in order to apply GSICS Correction to their applications.
⇒ Leave the choice of units up to each producer (agency)?
Variable std_scence_tb_bias (TB bias at standard scene):
⇒ Not extendable to VNIR, but we could define “standard_reflectivity_bias".

10. Impact of the VNIR product format definition
VNIR product = only the blend?
The previous decision was to have the individual results + blend
Visibility
But may make life more difficult (TBC) for the merged product…
Potential non-linearities (VNIR methods apply to different part of the dynamic range  calibration = equation with coeffs based on the methods results)
Long standing unanswered question = defining a more generic format for the correction coefficients  allowing for polynomial forms (with covariance matrixes for their uncertainties), as well as defining the form of the correction.
Comment by Chair GDWG: GDWG opinion is polynomial form is complex from data management point of view. In addition to that, we need to think about units of such polynomial variable. Correction coefficients have different units (e.g. offset term has W/m2/um/sr and liner term has W/m2/um/sr/count)  (probably) not allowed in CF Convention.

11. More questions…
Data organisation within the products ⇒ do we need to move to NetCDF4 Enhanced Data Model (e.g. “grouping” function)?
If variables are in scope for all channels, no need to organise internally the data in sub-blocks.
If not, the question may be discussed ⇒ User requirements?
GPPA to be revised to account for the update of the product life cycle when integrating more methods at different maturity levels.
Talking about maturity  should the maturity level of each method be a new variable?

12. Plotting tool: establishing requirements
Good tool to verify the product format implementation
In place for the IR
Nothing for VNIR  Need to converge on the product definition
But some prototype could be developed based on the current proposal (see next slide)  To be agreed with GDWG  

13. GSICS VNIR product (from #8l)
Currently one unique file for more methods
Global attributes  describe the dataset + its limitations + its origins (traceability, in particular to the version of the datasets used for the processing [reference + monitored instrument])
Dimensions:
1 record per file for Near-Real-Time products
Full time series to date for Re-Analysis products
Individual method results are all available  “Blend” = one additional method
Variables:
central_wavelength = ;
channel_name = "VIS06" ;
date = time_0…N;
method_name = "DCC“, "BLEND" ;
mon_official_offset = Level 1.5 offset_time_0…N;
mon_official_slope = Level 1.5 slope_time_0…N;
mon_offset = method_1_offset_time_0…N, … , method_M_offset_time_0…N;
mon_slope = method_1_slope_time_0…N, … , method_M_slope_time_0…N ;
validity_period = start_time_0…N, end_time_0…N;
weight_method = weight_method_1_time_0…N, … , weight_method_M_time_0…N;

14. Discussions at 2016 GSICS Annual Meeting
Inputs from GRWG
Bias monitoring would be a percentage departure from Day-1
Proposal: VNIR has it’s own web site, not combined with the IR
Current IR one: visualize a time series of TB temperature biases
Discussions at GDWG
Requirements (i.e. document) for the Plotting Tool would be prepared by GRWG. Then, GDWG would start updating the tool
The document is still under-preparation (?) by S. Wagner and M. Takahashi
Member agencies’ supports to GDWG activities are necessary

15. Plotting tool: establishing requirements
Other aspects may need to be defined before a consolidate plotting tool can implemented:
 VNIR product definition
 Product traceability
 GPPA + product development cycle
Is it easier to converge on intermediate products? (= for each individual method)
What could GDWG suggest?

16. Strategy for combining corrections for VIS/NIR+IR channels and plotting tool
Discussion