1. Analysis Ready Data (ARD) SEO Status Report
LSI-VC-3 Meeting
ESA-ESRIN, Frascati, Italy
March 20-21, 2017
Brian Killough
CEOS Systems Engineering Office
NASA Langley Research Center

2. SEO ARD Needs
The CEOS Open Data Cube initiative depends on ARD for its data source.
The SEO has worked with many international data users (GFOI and GEOGLAM) and ALL of them strongly desire ARD. None of them desire to pre-process data.
There are two reasons … minimize preparation time prior to analyses, and minimize science and technical knowledge required to properly process data into ARD.
In a “perfect World”, we would like satellite providers to routinely supply ARD. If that is not possible, then we desire to have open source processing tools and clear instructions.

3. What data is available from Google and Amazon?
Mostly Level-1 data, but some recent efforts to provide pre-processed Level-2 products by Google (Landsat and Sentinel-1).

4. ARD Status for Data Cubes

5. Sentinel-1 ARD
The SEO has been working on Sentinel-1 processing toward ARD. This ARD is focused on time series analyses of land and does not address In-SAR or the need for phase information.
The SEO has downloaded several S1 scenes over Vietnam (on right) and Colombia for testing. These scenes were Interferometric Wide Swath (IW), GRD format. There are two methods for processing. Use the ESA SNAP tool, or use a Python script.
The SEO installed the ESA SNAP tool and consulted several other sources for support … SNAP Sentinel-1 User Forum, Ben Lewis (GA), and Zheng-Shu Zhou (CSIRO).
The SEO selected the same set of steps as Ben Lewis (GA), as these were the most consistent with other users in the SNAP Sentinel Forum.
We have created a Python script that successfully processes GRD scenes into ARD.

6. Sentinel-1 Data Summary
The SEO has connected its COVE tool to the USGS Sentinel mirror archive. Sentinel-1 acquisitions are working in COVE. Sentinel-2 is coming soon …
Since April 2014 (~ 3 years), there are ~ 98,000 Sentinel-1 acquisitions.
77% of those acquisitions are available in all 3 product types … Raw, GRD, SLC.
99% of the acquisitions result in a GRD product (most common). 96% of the acquisitions result in an SLC product (includes phase information for interferometry). 82% result in a Raw product that could be processed to SLC and/or GRD.
GRD files are typically ~1 GB. SLC files are much larger and ~3-4 GB.

7. Sentinel-1 Processing Steps
(1) Orbit Updates - updates satellite ephemeris for improved geolocation (2) Remove Border Noise - removes processed artifacts at scene edges (3) Remove Thermal Noise – removes (4) Radiometric Calibration - converts raw data to backscatter intensity (beta-0 output) (5) Multilooking – produces a product with square pixel spacing. Not needed with terrain correction and most often used with SLC scenes. (6) Radiometric Terrain Correction - radiometric normalization (terrain flattening) using DEM data (gamma-0 output)
(7) Speckle Filter – removes noise but adds blurring to features and reduces resolution (8) Geometric Terrain Correction - orthorectification using DEM topography data (gamma-0 output in preferred grid projection) (9) Unit Conversion - convert from unitless intensity (DN) to commonly used decibels (dB)
The question is …. Which of these steps should we use?

8. Sentinel-1 Processing

9. Sentinel-1 Example (1 of 5)
Selected a sample image near Bogota, Colombia that includes mountains and plain regions.
IW mode, GRD format, 05-Sep-2016.
Ascending, Path=77, Frame=14

10. Sentinel-1 Example (2 of 5)
Unprocessed - VH Intensity
Lake: Laguna de Tota
Location: Sogamoso, Colombia
Step-3: Thermal Noise Removal
VH Intensity

11. Sentinel-1 Example (3 of 5)
Step-6: Radiometric Terrain Correction (terrain flattening) Gamma-0, VH Intensity
Step-4: Radiometric Calibration
Beta-0, VH Intensity

12. Sentinel-1 Example (4 of 5)
Step-7: Speckle Filtering (Refined Lee) Gamma-0, VH Intensity
Step-6: Radiometric Terrain Correction (terrain flattening) Gamma-0, VH Intensity

13. Sentinel-1 Example (5 of 5)
Step-8: Geometric Terrain Correction (orthorectification) Gamma-0, VH Intensity, WRS84 grid
Step-9: Unit Conversion (unitless intensity to dB) Gamma-0, VH Intensity (dB)

14. Next Steps – S1 ARD
What is the ARD for Sentinel-1 data that applies to the “majority” of users? Many of our users have a desire to use radar data for land change and land classification analyses without a desire for phase information or In-SAR.
Is there someone at ESA that routinely uses S1 data for land applications that we can contact to discuss these steps and get their opinions?
The only steps that differ among the 3 groups are: GRD border noise, thermal noise, multilooking, and speckle filtering
GRD border noise is a minor step and appears to be optional.
It is likely that thermal noise corrections are needed, as Google and CSIRO have selected this step.
Multilooking is not needed with terrain correction and not needed for GRD.
Speckle filtering is the most undecided step … many users say NO and others say YES. Not all users agree it is needed and some believe it “blurs” the images.
All of our Data Cube pilot countries want Sentinel-1 data. They are looking to us (CEOS) to define the ARD and help them produce that ARD and get the data into a cube.
There is a strong desire to use this data independently and interoperably with optical data (Landsat).

15. Sentinel-2 ARD
We believe the CARD4L for S2 is Surface Reflectance, similar to Landsat.
As of now, ESA does not supply SR products, but users can find the Sen2Cor algorithm to create the SR products. This code is NOT “open”.
Google Earth Engine does not allow users to process S2 to ARD on their system, since Sen2Cor is not “open source”.
ESA has issued a “tender” to procure an atmospheric correction algorithm for the future. When will this be complete? How will it impact the ARD processing steps or general approach to creating ARD?
Many of the Data Cube pilot countries desire S2 data. To date, we have not been able to to help them. Until we have a stable atmospheric correction algorithm and can generate an ingestor configuration, we have not supported S2 data for our users.
The ”Harmonized Landsat-Sentinel” (HLS) product (Masek, et.al.) uses a hybrid atmospheric correction, similar to Landsat. Is this something we could use for our Data Cube pilots for the S2 data? Is the HLS processing open source?
In order to make the greatest global use of S2 data, we need to resolve some of these issues.

16. ALOS-PALSAR ARD
The SEO has been working with Ake Rosenqvist to obtain ALOS-PALSAR data for testing.
Figure on right shows Landsat (RED), Sentinel-1 (BLACK) and ALOS-PALSAR (BLUE).
JAXA has now released free/open annual global mosaics (with forest / non- forest maps) at 25-m resolution for: JERS-1 (1996), ALOS-1 (2007, 2008, 2009, 2010) and ALOS-2 (2015). The ALOS mosaic is coming soon.
These products would be considered “level-3” as they are processed into ARD and include FNF maps.
Ake has developed an ALOS-PALSAR mosaic user interpretation guide (focused on GFOI) and is planning a similar guide for Sentinel-1. Our Data Cube users like a simple guide!

17. SPOT-5 ARD
Had a meeting with Stuart Phinn (Queensland Univ.) in November 2016 at the CEOS Plenary Meeting.
He has a process for atmospheric correction, terrain correction and NBAR for SPOT-5.
The SEO team was planning to send him a few SPOT-5 files over Kenya to process into Surface Reflectance (SR). We would select an area where we have Kenya data cube content.
Stuart would process the data into SR products. The SEO team would test them in QGIS (visual inspection against Landsat data). If the data looks promising, we will need to create a document for the processing steps, create a Python script for users to perform the same processing, and create a Data Cube ingestor.
Since we are very busy with other pilot projects, we need help from Stuart and CSIRO to move this forward.