Presentation is loading. Please wait.

Presentation is loading. Please wait.

Edward Armstrong, Jorge Vazquez, Andrew Bingham, Thomas Huang, Chris Finch, Charles Thompson, Tim Stough, Tim McKnight, and Cynthia Chen JPL PO.DAAC /

Published byPhilip Shaw Modified over 8 years ago

Similar presentations

Presentation on theme: "Edward Armstrong, Jorge Vazquez, Andrew Bingham, Thomas Huang, Chris Finch, Charles Thompson, Tim Stough, Tim McKnight, and Cynthia Chen JPL PO.DAAC /"— Presentation transcript:

1 Edward Armstrong, Jorge Vazquez, Andrew Bingham, Thomas Huang, Chris Finch, Charles Thompson, Tim Stough, Tim McKnight, and Cynthia Chen JPL PO.DAAC / GDAC California Institute of Technology 10 th GHRSST Science Team Meeting Santa Rosa, CA 1 June 2009

2 Introduction The Global Data Assembly Center (GDAC) Ingest and maintain interfaces to 32 GHRSST L2P/L3P/L4 datastreams from 10 RDACS. ~25 G/day > 5000 files/day Realtime quality checking for data and metadata granules Create FGDC metadata for daily collection granules Distribution via FTP/OPeNDAP/POET Maintain interfaces to the LTSRF for 30 day old data and metadata exchange GDAC web portal Work with RDAC data providers on new or updated products 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

3 Continued Maintain discovery and access interfaces. MMR has > 1.5 million records User and application support MODIS L2P production in collaboration with the OBPG and RSMAS Ancillary L2P filling Datacasting and other value added tools, services, and products 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

4 New GHRSST products Level 4 ABOM GAMSSA (28 km) and RAMSSA (9km) New version of AVHRR_OI, and AVHRR_AMSR_OI Level 2P NEODAAS regional L2P for AVHRR_17L, _18L UPA L2P for AATSR (ATS_NR_2P) Totals (32 active) 18 L2P 2 L3P 12 L4 Product table URL http://ghrsst.jpl.nasa.gov/GHRSST_product_table.html 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

5 Operations GDAC uptime 2 separate 2 day outages (down 100 hours; < 1%) Scheduled infrastructure maintenance RAID failure Some data provider outages (3 day outage of MODIS L2P) Latency No statistics compiled But no serious issues reported from users Coordination with LTSRF Close communication maintained between operational centers Have dealt with hiccups on a case-by-case basis 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

6 PO.DAAC Data Management and Archive System (DMAS) New ingest manager, new database, new discovery……NEW !! From the perspective of the RDACs nothing will change Improvements in scalability, throughput, load balancing, discovery, database storage, product validation, and metrics capturing. Ease of maintenance and operator supervision GHRSST datastreams have been tested with new system Fully operational by summer 2009 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

7 PO.DAAC Data Management and Archive System Service-oriented system Common metadata data model Metadata extraction and translation Data discovery Parallel and load-balanced data ingestion Dynamic deployment of ingest engines Storage management Validation State-driven Secure Portable Scalable

8 GHRSST Migration Adaptation MMR validation and translation Data file validation Scans local/remote locations for new data Integration with back-end RDAC cluster Inventory Full migration from existing MySQL database Port to use the new data model FGDC and Index generators Website Fully Operational End of June ’09

9 GDAC Distribution Statistics Number of distinct monthly users through April 2009 FTP/OPeNDAP/POET 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

10 Distribution Statistics Number of files distributed 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

11 Distribution Statistics Volume distribution 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

12 Product distribution – L2P By distinct number of users May 2007 – May 2009 aa 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

13 Product distribution – L4 By distinct number of users May 2007 – May 2009 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

14 XML TAG summary Current metadata shortcomings, two examples Limited data provenance, especially for L4 products No L2P orbit characterization… ISO roadmap Crosswalk from current GHRSST metadata model to ISO completed Preliminary UML diagrams based on ISO 19115, 19115-2, and 19139 completed Available: http://www.ghrsst-pp.org/GHRSST-PP-XML-Technical- Advisory-Group-(XML-TAG).htmlhttp://www.ghrsst-pp.org/GHRSST-PP-XML-Technical- Advisory-Group-(XML-TAG).html WMO metadata model compatibility 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

15 GDAC value added projects NASA MEaSURES proposal funded in collaboration with Univ. of Miami to generate global ultra high resolution blended products based on OI wavelet basis functions and motion compensation (PI Mike Chin, Arthur Mariano, Jorge Vazquez, Ed Armstrong).

16 10 th GHRSST Science Team Meeting, Santa Rosa, CA GDAC value added projects NASA Research to Operations proposal funded to improve existing SPORT SST composite products for the US east and west coasts (PI Jorge Vazquez and Gary Jedlovec). Collaboration with SPORT who already has customers in the National Weather Service and contacts in the oil and gas industry. Integration of GHRSST products for fisheries and marine ecosystem decision support and marine GIS (Dale Kiefer, USC and Ed Armstrong) Eddy detection (Charles Dong, UCLA) Seasonal SST fronts (Sara Purca, IMAEPE, Peru)

17 SST gradient and frontal probability maps Based on 4 km AVHRR Pathfinder and GHRSST NCDC AVHRR_OI data 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

18 SST gradient and frontal probability maps Based on 4 km AVHRR Pathfinder and GHRSST NCDC AVHRR_OI data 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

19 Virtual Ocean Data Center 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA NASA ACCESS Virtual Ocean Data Center (VODC) proposal funded to create infrastructure for transparent data and metadata harvesting and distribution using Object Orientated Data Technology (OODT) middleware (PI Rob Raskin, co-Is Chris Mattman, Sean McClease, Ed Armstrong). OODT runner up in NASA 2003 Software of the Year award “one-stop” Internet presence for searching, and extracting data from oceanographic data sources.

20 Search Architecture CAM-20

21 Search for Content (regular search page) CAM-21

22 10 th GHRSST Science Team Meeting, Santa Rosa, CA JPL Datacasting Model: an RSS-like feed for satellite data streams Client can subscribe to feed: “aware” on new data as soon as it is available Provider can package feed with metadata that consumer can filter on (e.g., location, cloud free pixels etc.) Datacasting is now operational on some GHRSST L2P data Version 2.0 client available

23 L2 subsetting Orbital L2 subsetting for regions of interest Based on NSIDC backtrack algorithm for granule query Prototyped on GHRSST L2P data Fall 2009 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

24 GHRSST L4 validation 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

25 Summary GDAC meeting all its fundamental operational and distribution requirements New data management architecture (DMAS) will make it easier to expand to the increasing volumes of GHRSST datastreams Low downtime Users reporting no or little issues with access Improved tools and services on the horizon Discovery improvement with VODC and “new” MMR Implement L2 subsetting More datacasting streams and browse images Improvements Metrics capturing (“casting” and “harvesting”) ISO metadata implementation Product documentation Applications and products UHR SST L4 products Supporting directly or indirectly a number of user applications 1 June 200910 th GHRSST Science Team Meeting, Santa Rosa, CA

Download ppt "Edward Armstrong, Jorge Vazquez, Andrew Bingham, Thomas Huang, Chris Finch, Charles Thompson, Tim Stough, Tim McKnight, and Cynthia Chen JPL PO.DAAC /"

Similar presentations

Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO.DAAC Ed Armstrong, Jorge Vazquez and Andrew Bingham

Status of the GHRSST GDAC (Global Data Assembly Center) at the JPL PO.DAAC Ed Armstrong, Jorge Vazquez and Andrew Bingham
Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang Distribution.

Ed Armstrong, Jorge Vasquez, Andrew Bingham, Sue Heinz, Alex Cervantes, Qui Chau, Tim McKnight, Wendy Zhang Distribution.
National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Pasadena, California Metadata-Centric Discovery.

National Aeronautics and Space Administration Jet Propulsion Laboratory California Institute of Technology Pasadena, California Metadata-Centric Discovery.
ESIP Federation Meeting 15-18 July 2008 Data and Metadata Standards for GHRSST: present to future ESIP Federation Meeting 15-18 July 2008 Univ. of New.

ESIP Federation Meeting July 2008 Data and Metadata Standards for GHRSST: present to future ESIP Federation Meeting July 2008 Univ. of New.
6-7 May 2002 CEOS GRID Workshop 1 “CEOS GRID Proposals” roundtable: Introduction ESRIN, 6-7 May 2002 CEOS Workshop on GRID.

6-7 May 2002 CEOS GRID Workshop 1 “CEOS GRID Proposals” roundtable: Introduction ESRIN, 6-7 May 2002 CEOS Workshop on GRID.
GHRSST Data Access Tutorial GHRSST Data Access How to Access GODAE High Resolution SST Products from the GDAC and LTSRF Kenneth.

GHRSST Data Access Tutorial GHRSST Data Access How to Access GODAE High Resolution SST Products from the GDAC and LTSRF Kenneth.
Thomas Huang PO.DAAC Software System Engineer Jet Propulsion Laboratory California Institute of Technology These activities were carried out at the Jet.

Thomas Huang PO.DAAC Software System Engineer Jet Propulsion Laboratory California Institute of Technology These activities were carried out at the Jet.
Virtual Quality Screening Service (VQSS): Improving the application of quality information NASA funded Advancing Collaborative Connections for Earth System.

Virtual Quality Screening Service (VQSS): Improving the application of quality information NASA funded Advancing Collaborative Connections for Earth System.
GCMD/IDN STATUS AND PLANS Stephen Wharton CWIC Meeting February19, 2015.

GCMD/IDN STATUS AND PLANS Stephen Wharton CWIC Meeting February19, 2015.
Web Services for Earth Science Data Edward Armstrong, Thomas Huang, Charles Thompson, Nga Quach, Richard Kim, Zhangfan Xing Winter ESIP 2014 Washington.

Web Services for Earth Science Data Edward Armstrong, Thomas Huang, Charles Thompson, Nga Quach, Richard Kim, Zhangfan Xing Winter ESIP 2014 Washington.
Aquarius Project Status 7 th Aquarius/SAC-D Science Team Meeting 11 April 2012.

Aquarius Project Status 7 th Aquarius/SAC-D Science Team Meeting 11 April 2012.
UWG 2013 Meeting PO.DAAC Data Stewardship Status.

UWG 2013 Meeting PO.DAAC Data Stewardship Status.
1 A National Virtual Specimen Database for Early Cancer Detection June 26, 2003 Daniel Crichton NASA Jet Propulsion Laboratory Sean Kelly NASA Jet Propulsion.

1 A National Virtual Specimen Database for Early Cancer Detection June 26, 2003 Daniel Crichton NASA Jet Propulsion Laboratory Sean Kelly NASA Jet Propulsion.
OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger Life Cycle Architecture Review.

OOI CI LCA REVIEW August 2010 Ocean Observatories Initiative OOI Cyberinfrastructure Architecture Overview Michael Meisinger Life Cycle Architecture Review.
Discovery and Web Services in Support of SST Datasets at the PO.DAAC Edward Armstrong, Jorge Vazquez Toshio M. Chin, Charles Thompson Jet Propulsion Laboratory/California.

Discovery and Web Services in Support of SST Datasets at the PO.DAAC Edward Armstrong, Jorge Vazquez Toshio M. Chin, Charles Thompson Jet Propulsion Laboratory/California.
IIIrd SeaSearch Full Group Meeting Limassol, Cyprus Tuesday May 25-27, 2004 SeaDataNet-Design Study proposal.

IIIrd SeaSearch Full Group Meeting Limassol, Cyprus Tuesday May 25-27, 2004 SeaDataNet-Design Study proposal.
Other Related Activities Gary Jedlovec transitioning unique NASA data and research technologies to operations 12 17 22 27 Single image MODIS Composite.

Other Related Activities Gary Jedlovec transitioning unique NASA data and research technologies to operations Single image MODIS Composite.
Students: Anurag Anjaria, Charles Hansen, Jin Bai, Mai Kanchanabal Professors: Dr. Edward J. Delp, Dr. Yung-Hsiang Lu CAM 2 Continuous Analysis of Many.

Students: Anurag Anjaria, Charles Hansen, Jin Bai, Mai Kanchanabal Professors: Dr. Edward J. Delp, Dr. Yung-Hsiang Lu CAM 2 Continuous Analysis of Many.
Opendap dev - meeting, Boulder, Feb 2007 OPeNDAP infrastructure in European Operational Oceanography T Loubrieu (IFREMER) T Jolibois (CLS)

Opendap dev - meeting, Boulder, Feb 2007 OPeNDAP infrastructure in European Operational Oceanography T Loubrieu (IFREMER) T Jolibois (CLS)
AN ENHANCED SST COMPOSITE FOR WEATHER FORECASTING AND REGIONAL CLIMATE STUDIES Gary Jedlovec 1, Jorge Vazquez 2, and Ed Armstrong 2 1NASA/MSFC Earth Science.

AN ENHANCED SST COMPOSITE FOR WEATHER FORECASTING AND REGIONAL CLIMATE STUDIES Gary Jedlovec 1, Jorge Vazquez 2, and Ed Armstrong 2 1NASA/MSFC Earth Science.

Similar presentations

Ads by Google