1. Microwave Radiometer Processing for Carbon in the Arctic Reservoirs Experiment (CARVE)
Nick Steiner1
1 - Advanced Science Research Center, City University of New York, New York, NY
NSIDC DAAC UWG Meeting
August 9-10
Boulder, CO

2. Overview Study Area: CARVE Domain
Data Types: Radiometer Observations (AMSR-E,AMSR-2, SSM/I)
Data Workflow
Challenges/friction points
Recommendations

3. CARVE -Carbon in Arctic Reservoirs Vulnerability Experiment
The carbon budget of Arctic ecosystems are not well known
Study how greenhouse gas exchange is linked to fundamental elements across the complex Arctic biological-climatologic- hydrologic systems
Aircraft measurements of important greenhouse gases on local to regional scales in the Alaskan Arctic
Related remote sensing products (hydrologic/climatologic surface controls)
Surface hydrology and inundation
Freeze/thaw cycles

4. Data Types - Combined Passive Microwave Radiometers
Satellite data from multiple instruments to create continuous product
Gaps in coverage overlapping experiment
Combined to create continuous observational record
Multi-frequency observations needed for processing
Many channels to coordinate
Swath to grid processing
Account for the physical characteristics of instrument
Antennae footprint size. shape

5. Data Access - Overview
Question Driven Data Access – Multi-frequency observations to focus on hydrologic change in different terrain components
Multiple passive microwave radiometer channels and pass timing (6 final product)
Combined with observations from aircraft and station measurements of CO2, CH4
Continuous data pull during daily experiment flights
Data product browse used in experimental design – flight lines
Near real-time for observations capturing freeze/thaw and snowmelt transitions.

6. Data Workflow – Friction Points/Challenges
Three Different Satellite instruments – AMSR-E (validation), SSM/I (gaps in data record ) and AMSR-2
High Data Volume - 3 x 120 swath level products daily/satellite
Basic Data Access – pulled from FTP/Filesystem (no API)
Need way to obtain swaths programmatically
Solutions – Strip metadata and ingest locally
Upload into local databases with appropriate fields (versioning, time, location in file/system)
Object-Relational Mapping (Python-SQLAlchemy)
Query return object with read/write functions for associated data (HDF5)

7. Data Workflow – Friction Points/Challenges
Preprocessing
Filter bad scan-lines
RFI Correction
Water-body and coastline filtering
Swath to grid aggregation
Subset raw observations by location and time (ascending/descending)
Gridding based on physical parameters of the instrument
Footprint matching - channel specific antennae
Overlap - by design
Grid-spacing
Solution: Multidimensional Database (SciDB)

8. Recommendations/Ideas
Python notebook – Recreate / stage figure with graphing scripts
Allow user to manipulate data analysis
Libraries for Common Tasks (Community Based)
Programmatic access to data (Download  Archive)
Selecting and Filtering
Read/Write and Visualize
Reprojection and gridding
Packaged collections of Remote Sensing libraries
Numpy, Sci-Kit Learn, Pandas, GDAL
Anaconda (python)
Virtual machines
Data/Software kits