Analysis Ready Data July 18, 2016 John Dwyer Leo Lymburner Landsat Project Scientist, USGS dwyer@usgs.gov +1 605 594-606 Leo Lymburner Land Cover Team Lead, GA Leo.Lymburner@ga.gov.au +61 2 6249 9587

Why is ARD critical? Advantages for producers Advantages for end users Facilitates expedient access More efficient data management Traceability of data provenance Enables targetted data retrieval and utilisation – reduction in data transfer volumes Advantages for end users Enable automated analysis Support sensor interoperability Less effort on preprocessing – more time to generate higher level products

In simplest terms analysis ready data should be… Measurements of the Earths surface from the same sensor and the same wavelength are comparable in space and time …and preferably… Measurements from the different sensors and the same wavelength are comparable in space and time

Geometric alignment Different origins Sensor specific acquisition orientation Grids don’t line up

Spatial comparability of different collections Characterise the geometric accuracy of each sensor collection

The challenge of different spatial resolutions Sentinel 2 Landsat MODIS

per pixel metadata Time PQ flagged per band -Saturation PQ flagged observation -Cloud

Spectral comparability ARD must allow users To keep track of the sensor That acquired a measurement Sensor 1 ‘Blue Band’ Sensor 2 ‘Blue Band’ Sensor 3 ‘Blue Band’

Community Established Terms NASA EOS Processing Levels Level-0: Reconstructed, unprocessed instrument and payload data Level-1: Data that have been processed to sensor units Level-2: Derived geophysical variables at the same resolution and location as Level 1 source data. Level-3: Variables mapped on uniform space-time grid scales, usually with some completeness and consistency. Level-4: Model output or results from analyses of lower-level data (e.g., variables derived from multiple measurements).

Hierarchical Definition of ARD ARD Level Data Type Example / Description NASA EOS Threshold Target Break- through   ARD-Specialist corrections Scene-based calibrated radiances Level-1T Scaled DNs 2G/1 X Scene-based TOA reflectance Level-1T TOA reflectance ARD-Specialist analysis Scene-based surface reflectance Level-1T atmospherically corrected only 2G/2 Scenes of Nadir view angle corrected BRDF corrected atmospherically corrected surface reflectance Seamless atmospherically corrected surface reflectance Gridded surface reflectance with corresponding pixel quality flags (could be from either scene-based SR options above) 3 ARD-utilisation by non-EO community Surface reflectance composites Best available pixel composites per epoch (could be from either scene-based SR options above) 4 A comprehensive framework to highlight the risk of people misinterpreting what ARD is/means Need to narrow the focus to GA/USGS ARD collections Surface Temperature is next on the list…

Hierarchical Definition of ARD ARD Level Data Type Example / Description NASA EOS Threshold Target Break- through   ARD-Specialist corrections Scene-based calibrated radiances Level-1T Scaled DNs 2G/1 X Scene-based TOA reflectance Level-1T TOA reflectance ARD-Specialist analysis Scene-based surface reflectance Level-1T atmospherically corrected only 2G/2 Scenes of Nadir view angle corrected BRDF corrected atmospherically corrected surface reflectance Seamless atmospherically corrected surface reflectance Gridded surface reflectance with corresponding pixel quality flags (could be from either scene-based SR options above) 3  TOA SR  NBAR ARD-utilisation by non-EO community Surface reflectance composites Best available pixel composites per epoch (could be from either scene-based SR options above) 4 A comprehensive framework to highlight the risk of people misinterpreting what ARD is/means Need to narrow the focus to GA/USGS ARD collections Surface Temperature is next on the list…

Proposed LSI-VC/CEOS definition of ARD Geophysical measurements that are comparable in space and time with sufficient per-pixel (observation) metadata to enable users to select ‘observations of interest’ as input into their analyses. Or in USGS/GA terms ‘surface reflectance in conjunction with PQ/QA bands’ NASA-EOS level 2 data with metadata Collection level (e.g. CPF, GCP specification) Record level ( e.g. time of acquisition) Pixel level (e.g. PQ/QA band) The USGS ARD defintiion needs to updated to include additional QA bits associated with surface reflectance - interpolated aerosols, updated cloud information. John will add this info to the PQ//QA tables later in the deck; will try to get this included by the dry-run, but certainly by LA meeting.