1. Moderate Resolution Sensor Interoperability (MRI) Initiative
Committee on Earth Observation Satellites
Moderate Resolution Sensor Interoperability (MRI) Initiative
Gene Fosnight, USGS
2017 CEOS Chair Initiative
USGS EROS – Sioux Falls, USA
16th May 2017

2. Moderate Resolution Sensor Interoperability (MRI) Initiative
This initiative will include effort towards making optimal use of the increasing number of data streams available in the moderate resolution class, with a focus for on Landsat/Sentinel-2.
As higher level, multi-sensor, time series products are developed, the integration of these products requires verification and validation of their interoperability, such as
when are these products interoperable,
when are they not interoperable, and
under what conditions?

3. 2017 Deliverables
Develop a MRI Framework paper for moderate (10-100m) resolution interoperability, identifying multi-sensor interoperability concepts that need to be addressed for successful implementation of multi-sensor interoperable time series
Address factors such as radiometry, geometry, data formats, browse, metadata, data access, metrics, and reporting
A Landsat/Sentinel-2 interoperability case study document utilizing the MRI Framework, including lessons learned and best practices identified through the implementation and use of the Harmonized Landsat Sentinel- 2 (HLS) Surface Reflectance product

4. Deliverable #1: MRI Framework (Concept)
Interoperability solutions follow two paths:
Changes to products or post processing methodologies to create interoperable products - Harmonization
radiometric cross calibration to standard references
acceptance of common geographic reference grid and DEMs
compatible atmospheric models
Accommodation to inherent differences between products – Homogenization
pixel size
spectral response curves
available bands

5. Deliverable #1: MRI Framework (Concept)
Component
Measure
Threshold (Minimum)
Target (Goal)
Verification Results
Next Steps
General Metadata
Geometric Specifications & Accuracy
Radiometric Specifications & Accuracy
Temporal Specification
Per Pixel Metadata
Cloud Cover and Shadow
Land, Water, Snow/Ice and Vegetation Pre- maps
DEM and Terrain Shadow
Solar and View Angle
Data Quality and No Data
Atmospheric Model Inputs
Radiometry
Measurements
Solar and View Angle Corrections
Atmospheric Corrections
Spectral Band Difference Corrections
Geolocation
Geometric Corrections
Resampling

6. Interoperability Check List
Do the products meet the CARD4L product family specification?
Determine common map projection, origin and pixel size to meet user requirements. What are the geometric parameters of the merged product?
Extract bands from each product appropriate for a merged product. Bands may be available for some, but not for other source data sets. What bands are used in the merged product for each sensor?
Does the joint relative geometric accuracy among the products meet the interoperability criteria for the user application? What is the individual and joint geometric accuracies?
If the relative geometric accuracy is not met, is it possible to perform an image-to-image registration? What methods are used to meet the user’s geometry criteria?
Are the products all from the same product family, for example Surface Reflectance?
If the products are not from the same product family, is the necessary atmospheric and geometric per-pixel metadata and methodologies available to create a consistent merged product. What atmospheric and BRDF models are applied?
Does the radiometric accuracy of the input data sets meet the interoperability criteria for the user application? What are the radiometric accuracies of the products?
Are spectral band adjustment factors available to correct significantly different spectral response curves? What are the factors and how are they applied?

7. Deliverable #2: Landsat/Sentinel-2 Interoperability Case Study
The NASA Harmonized Landsat 8 Sentinel-2 (HLS) product generation addresses many of the interoperability issues laid out in the MRI Framework.
The EC vegetation dynamics monitoring project using the HLS products in conjunction with the phenology algorithm will be the primary interoperability use case.
Case study evaluation will utilize the MRI Framework and address multi-sensor interoperability concepts.
Beyond HLS, agencies and users will be surveyed to obtain a more comprehensive list of ongoing efforts to use Landsat and Sentinel-2 data together.
These projects can provide a basis for understanding “lessons learned” and best practices.

8. Harmonized Landsat Sentinel-2 (HLS) Project
Merging Sentinel-2 and Landsat data streams can provide 2-3 day global coverage
Goal is “seamless” near-daily 30m surface reflectance record including atmospheric corrections, spectral and BRDF adjustments, regridding
Project initiated as collaboration among GSFC, UMD, NASA Ames

9. MRI: Relationships

10. Deliverable #2: Landsat/Sentinel-2 Interoperability Case Study
HLS is a merged Landsat 8 Sentinel-2 Surface Reflectance product
Machine readable metadata
Resampled to 30-meter Sentinel-2 reference
Cloud masks
View and solar angle BRDF corrections
Atmospheric correction
Band pass adjustment
Products available for over 747 MGRS tiles and covering 63 regions
Document Best Practices and Lessons Learned through the HLS product generation and HLS Phenology Use Case

11. Cross-calibration issue or spectral band adjustment not optimal?
HLS Example: Radiometry
South Africa
Cross-calibration issue or spectral band adjustment not optimal?
NDVI

12. HLS Example: Crop Phenology
South Africa
Southern France
NDVI
NDVI time series from Landsat (red) and Sentinel-2a (blue) reflectance values show detailed crop phenology

13. HLS Example: Within Season Monitoring

14. MRI Team Co-leads Team Members Gene Fosnight USGS
USGS chair team co-lead, Landsat, LSI-VC, SDCG
Cindy Ong
CSIRO
WGCV co-lead, imaging spectroscopy
Richard Moreno
CNES
WGISS co-lead, Copernicus
Jeff Masek
NASA
Landsat Sentinel-2 Case study, LSI-VC
Zoltan Szantoi
JRC
Adam Lewis GA
LSI-VC, FDA, CARD4L
Paul Briand
CSA
LSI-VC, GEOGLAM, SAR
Yves Crevier
SDCG, GEOGLAM, SAR
Brian Killough
SEO, LSI-VC, SDCG, FDA, CARD4L
Debajyoti Dhar
ISRO
Optical/SAR data fusion
Takeo Tadono
JAXA
LSI-VC, CARD4L, SAR
Amanda Regan EC
User needs for satellite constellations
Nigel Fox
UKSA
WGCV IVOS
Kurt Thome
WGCV
Andy Mitchell
WGISS
Kerry Sawyer
NOAA
Represent SIT vice chair
Kevin Gallo
LSI-VC, LPCS data integration
Eric Wood
USGS
Represent USGS Chair Team

15. Roadmap Conduct Kickoff telecon (March 2017)
Discuss MRI at LSI-VC-3 (March 2017) and at WGISS (April 2017)
Identify Landsat/Sentinel-2 case study (early April 2017)
Present MRI at SIT-32 (April 2017 )
Distribute working copy of MRI Framework document (May 2017)
Discuss MRI at WGCV (May 2017)
Distribute final draft of MRI Framework document for review (late July )
Provide HLS product interoperability evaluation and phenology use case for review (late August 2017)
Present initial case study results at LSI-VC-4 and SIT TW (September 2017)
Present final MRI Framework and case study results at CEOS Plenary (October 2017)
Present proposed way forward at CEOS Plenary (October 2017)

16. Organization & CEOS Interactions
The Moderate Resolution Sensor Interoperability (MRI) initiative is an activity under the Land Surface Imaging Virtual Constellation (LSI-VC)
In particular, MRI team is also working closely with the LSI-VC CEOS Analysis Ready Data for Land (CARD4L) team
CARD4L are satellite data that have been processed to a minimum set of requirements and organized into a form that allows immediate analysis with a minimum of additional user effort and interoperability both through time and with other datasets.
CARD4L provides product family specifications, where examples are top-of- atmosphere reflectance, Surface Reflectance, Surface Temperature, SAR sigma or SAR gamma naught. MRI provides the basis for combining and comparing multiple products.
LSI-VC is working in partnership with the Working Group on Calibration & Validation (WGCV) and the Working Group on Information Systems and Services (WGISS)

17. MRI General Metadata Items MRI concepts MRI Alternatives
Coordinate Reference System
Different pixel sizes, origins and projections require the data be spatially resampled. The greater the differences in pixel sizes the greater the effect on radiometry.
Reproject to either the smaller or larger pixel size depending upon application requirements and common origin and projection.
Geometric correction
Different and less accurate reference grids will cause greater uncertainty in spatial alignment of pixels.
Minimize absolute error of individual reference grids. When possible share reference grids across sensors.
Geometric accuracy
Spatial RMSE distributions for each sensor are needed to establish the joint distribution for multi-sensor time series.
Reduce RMSE to the theoretical minimum per sensor in relationship to reference grid.
Spectral bands
Different bands between sensors require methodologies that can adapt to band availability.
Different bands are inherent differences. When possible provide the user community the richest set of alternatives, even when it creates a discontinuity in the time series record for some bands.
Spectral response curves
Different spectral response curves exist for similar bands between sensors causing land surface dependent variability.
Spectral Band Difference Factors (SBAF) can be used to adjust the radiometry to accommodate differences under many conditions.
Radiometric Accuracy
Biases and uncertainty need to be minimized between sensors using spectrally uniform references.
Cross calibration using uniform reference surfaces can minimize overall biases between sensors.
Revisit time & lifetime
Multi-sensor time series extents time series and increases the temporal density at the cost of increase variability in the signature.
Longer time series permit long-term change to be monitored.
More frequent revisit times increases the probability that land cover change can be detected by decreasing the sampling interval and by increasing the probability of cloud free data.

18. MRI Per-Pixel Metadata
Items
MRI concepts
MRI Alternatives
Clouds
Verified and validated cloud masks are needed to estimate radiometric contamination for specific and known cloud characteristics. Known confusion such as with snow/ice needs to be quantified.
Verify and validate cloud algorithms. Share algorithms to minimize variability when appropriate. Different band availability will cause differential results. These differences need to be documented
Cloud Shadow
Verified and validated cloud shadow masks are needed to estimate radiometric contamination for specific and known cloud characteristics.
Known confusion such as with water needs to be quantified. Cloud shadows contain spectral information that can be used within many applications. Adaptive algorithms can use the masks appropriately.
Land/water mask
Simple premapping of land and water provides useful information for other radiometric corrections.
Consistently handle differential corrections over land and water.
Snow & Ice masks
Premapping of snow and ice assists in cloud cover assessment.
Time series are used both in the detection of clouds and in the monitoring of snow and ice. Verified and validated uncertainty estimates improve both applications.
DEM
The required accuracy of the DEM is dependent upon the corrections implemented, swath width and pixel size.
Share DEMs when possible both among operational agencies and with users. Requirement are highly variable.
Terrain Shadow mask
Terrain shadow masks are needed to estimate radiometric contamination associated with shadows. Known confusion such as with water needs to be quantified.
Terrain shadows are particularly important for mountainous areas, wide swaths and for SAR sensors. Terrain shadows contain spectral information that can be used within many applications. Adaptive algorithms can use the masks appropriately.
Illumination and Viewing geometry
Solar illumination angles are needed for reflectance calculations. Solar and View angles are needed for BRDF related corrections including terrain illumination corrections.
Define minimum correction versus full BRDF corrections. Issues opportunities, uncertainties.
Data Quality
No data, saturated, contaminated, terrain occlusion pixels need to be identified.
Distinguish between no data and contaminated data. Document how these pixels are handled during resampling.

19. MRI Measurement Data Items MRI concepts MRI Alternatives Measurements
Absolute calibrated measurement units with or without corrections below
Minimum requirement is reflectance calibrated and validated to a known absolute source. Atmospheric, solar angle and viewing angle, SBAF corrections are usually needed to minimize variability and uncertainty to create an interoperable product.
Measurement nomalisation
Radiometry viewed through time is significantly impacted by variation in solar and viewing angles
Solar and viewing angle corrections compensate for both within and between sensor variability.
Aerosol, water vapor and Ozone corrections
Different atmospheric models can introduce significant between sensor variability.
Atmospheric model must either be shared or validated and verified to common reference. Available bands and ancillary data will impact quality of corrections
SBAF corrections
Different spectral response curves will introduce biases between instruments.
Spectral Band Adjustment Factors (SBAF) compensate for different spectral response curves and will be application and surface type variable.

20. MRI Geolocation Items MRI concepts MRI Alternatives
Geometric Corrections
Mis-registration between images and between reference databases introduce variability in the radiometry measurements
Minimize misregistration through orthorectification and precision registration to a controlled reference grid
Resampling
The number and type of spatial resampling will impact the radiometric signal
Minimize the number of resampling operations and consider impact objects such as contaminated pixels including clouds.

21. Interoperability Case Study
Where does CARD4L end and interoperability begin?
These steps standardise the data to be ‘analysis ready’. They are consistent with CARD4L but exceed the threshold specification (BRDF)
Subsequent steps are specifically blend these two data streams.
The method of blending is not unique; other approaches are also possible.