DMAC: Infrastructure to enable delivery of IOOS ® information Presentation to the IOOS Advisory Committee Derrick Snowden, System Architect, U.S. IOOS.

Slides:

Advertisements

Similar presentations

ECOOP Data Management System (T2.2/WP2) Declan Dunne 13 th February 2008, Athens.

Advertisements

The Model Output Interoperability Experiment in the Gulf of Maine: A Success Story Made Possible By CF, NcML, NetCDF-Java and THREDDS Rich Signell (USGS,

DMAC ST and the Activities of the IOOS PO Derrick Snowden DMAC Steering Team

Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.

Crossing the Digital Divide

The SouthEast Coastal Ocean Observing SECOORA Meeting Regional Association (SECOORA) June 11-12, 2007 SECOORA Data Management and Communications SECOORA.

Integrated Ocean Observing System Data Management and Communications March 2004 The US Integrated Ocean Observing System (IOOS) Plan for Data Management.

User Needs for data Oil Spills Search & Rescue IOOS Demo Project Eoin (Owen) Howlett ASA Inc. Narragansett, RI SECOORA/SEACOOS Workshop.

A Distributed Approach to Ocean, Atmosphere & Climate Model Data Interoperability Rich Signell USGS Woods Hole, MA COAWST Training Workshop Woods Hole,

Information Technology for Ocean Observations and Climate Research TYKKI Workshop, December 9-11, 1998, Tokyo, Japan Nancy N. Soreide NOAA Pacific Marine.

Serving society Stimulating innovation Supporting legislation Meeting on OGC Sensor Observation Service (SOS) for INSPIRE Mickael.

The use of standard OGC web services in integrating distributed model, satellite and in-situ datasets Alastair Gemmell Jon Blower Keith Haines Environmental.

A Super-Regional Modeling Testbed for Improving Forecasts of Environmental Processes for the U.S. Atlantic and Gulf of Mexico Coasts Cyberinfrastructure.

From Data Integration to Integrated Solutions Tom Shyka, GMRI Paul Currier, NH DES Water Quality Workshop MACOORA - NERACOOS -SECOORA IOOS, USGS, EPA,

WPS Application Patterns at the Workshop “Models For Scientific Exploitation Of EO Data” ESRIN, October 2012 Albert Remke & Daniel Nüst 52°North Initiative.

GADS: A Web Service for accessing large environmental data sets Jon Blower, Keith Haines, Adit Santokhee Reading e-Science Centre University of Reading.

Implementation of Model Data Interoperability for IOOS: Successes and Lessons Learned Rich Signell USGS Woods Hole, MA / NOAA Silver Spring USA Model Data.

Observing System Monitoring Center Integrating data and information across observing system networks.

Geospatial Systems Architecture Todd Bacastow. GIS Evolution

Unidata’s TDS Workshop TDS Overview – Part II October 2012.

General Assembly Meeting – SA Session | Crete Oct SA 13.1 NEMO-SN1 – WESTERN IONIAN SEA GEOPHYSICS, OCEANOGRAPHY, AND DERIVED PRODUCTS SA starting.

Testing the US Integrated Ocean Observing System Data Discovery and Distribution Infrastructure with Real-World Problems Derrick Snowden 1, Rich Signell.

2 3 ROMS/COAWST NcML file 4 5 Exploiting IOOS: A Distributed, Standards-Based Framework and Software Stack for Searching, Accessing, Analyzing and.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

IOOS Data Management Integration Standards Plans in the Northeast Eric Bridger (GMRI) Sep

THREDDS Data Server Ethan Davis GEOSS Climate Workshop 23 September 2011.

IOOS DMAC Meeting Regional Association Discussion IOOS PO May 2015.

Enhancements to a Community Toolset for Ocean Model Data Interoperability: Unstructured grids, NCTOOLBOX, and Distributed Search Rich Signell (USGS), Woods.

Tools in Support of a National DMAC Derrick Snowden NERACOOS/ODP Annual Meeting 26 Sep 2012.

Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.

NcBrowse A Graphical netCDF/OPeNDAP Browser Donald Denbo 1 & John Osborne 2 1 UW/JISAO-NOAA/PMEL, 2 OceanAtlas Software

Unidata and Oceanography Through the Ages Rich Signell USGS Coastal and Marine Science Center Woods Hole, MA & NOAA Integrated Ocean Observing System (IOOS)

IOOS Model Data Interoperability Design ROMS POM WW3 WRF ECOM NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF)

1 NOAA IOOS Program Data Integration Framework (DIF) Project Overview Adapted from a brief to the NOAA Data Management Committee August 6, 2008 by Jeff.

Service Oriented Architectures Presentation By: Clifton Sweeney November 3 rd 2008.

GEO-IDE Community Building and Best Practices Process GEO-IDE Standards Process Denver Post, Sunday, 9 April 2006.

Information, data servers and data products for the Pacific regional association J. Potemra briefing to Z. Willis, UH Manoa, July 2009.

THEME[ENV ]: Inter-operable integration of shared Earth Observation in the Global Context Duration: Sept. 1, 2011 – Aug. 31, 2014 Total EC.

IOOS Modeling Testbed Cyberinfrastructure Rich Signell, USGS, Woods Hole, MA IOOS-RA-Briefing, Feb 14, 2012.

IOOS Coastal Ocean Modeling Testbed (COMT) Cyberinfrastructure Oceans 12 Becky Baltes, IOOS Liz Smith, SURA Rich Signell, USGS Eoin Howlett, Kyle Wilcox,

1 Zdenka Willis IOOS Program Director March 20, 2015 Zdenka Willis IOOS Program Director March 20, 2015.

Geospatial Systems Architecture Todd Bacastow. Views of a System Architecture Enterprise Information Computational Engineering Technology.

U.S. Integrated Ocean Observing System (IOOS ® ) IOOS ® Biological Observations Data Project A Multi-Agency Effort to Enable Access to Biological Observations.

IOOS Data Services with the THREDDS Data Server Rich Signell USGS, Woods Hole IOOS DMAC Workshop Silver Spring Sep 10, 2013 Rich Signell USGS, Woods Hole.

Air Quality Data Services: Application of OGC specifications Air Quality Data: Multi-dimensional, multi-source, multi-format Point observations are collected.

Unidata’s TDS Workshop TDS Overview – Part I July 2011.

Kevin O’Brien University of Washington/JISAO NOAA/PMEL The Observing System Monitoring Center Steve Hankin, PMEL Ted Habermann, NGDC David Neufeld, NGDC.

Ed Armstrong – PI Luca Cinquini Chris Mattmann NASA Jet Propulsion Laboratory Frank O’Brien Zach Siegrist System Science Applications, Inc. 18 July 2012.

NOAAServer: Unified access to distributed NOAA data Ernest Daddio, NOAA/ESDIM Steve Hankin, NOAA/PMEL Donald Denbo, NOAA/PMEL/JISAO Nancy Soreide, NOAA/PMEL.

IOOS Regional Data Management Workshop April 27 & 28, 2011 Silver Spring, MD COLLABORATION TO DATE Charles Alexander 1, Jeff de La Beaujardière 1, Arthur.

1 of 37 Next Steps SOS DescribeSensor implementation (SensorML) Testing/evaluation/refinement of existing work Build or borrow needed components … Reference.

Jeff de La Beaujardière, PhD NOAA Sr System Architect, IOOS Program Office DAARWG Meeting Silver Spring, Integrated Ocean Observing System (IOOS)

TECHNOLOGY GUIDE THREE Emerging Types of Enterprise Computing.

U.S. INTEGRATED OCEAN OBSERVING SYSTEM DATA ACCESS SERVICES Coastal Geotools: Advances in Federal Data Access Services Hassan Moustahfid, Chris Duncombe.

Data Stewardship at the NOAA Data Centers Sub Topic - Value Added Products ESIP Federation Meeting, Washington, DC January 6-8, 2009.

Catalog-driven workflows using CSW Rich Signell, USGS, Woods Hole, MA, USA Filipe Fernandes, SECOORA, Brazil Kyle Wilcox, Axiom Data Science, Wickford,

Windows Azure poDRw_Xi3Aw.

Ocean Observatories Initiative Serving Ocean Model Data on the Cloud M. Meisinger, C. Farcas, E. Farcas, C. Alexander, M. Arrott, J. de La Beaujardière,

Access to IOOS Data Relevant to OOI Kathleen Bailey NOAA/NOS/IOOS January 6, 2016.

Maracoos DMAC and Asset Map June Maracoos DMAC Principles Data is served and integrated using open, interoperable data standards recommended by.

1 Zdenka Willis IOOS Program Director March 20, 2015 Carl Gouldman IOOS Deputy Director May 8, 2015.

Rich Signell Roland Viger Curtis Price USGS Community for Data Integration Feb 15, 2012.

1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)

The NOAA Operational Model Archive and Distribution System NOMADS CEOS-Grid Application Status Report Glenn K. Rutledge NOAA NCDC CEOS WGISS-19 Cordoba,

Interoperability Day Introduction Standards-based Web Services Interfaces to Existing Atmospheric/Oceanographic Data Systems Ben Domenico Unidata Program.

NOAA IOOS SOS Implementations in 2008 Jeff de La Beaujardière, PhD NOAA IOOS Program DIF Sr Systems Architect.

NOAA EDMC Ocean Observatories Initiative Cyberinfrastructure Karen Stocks OOI CI Data Curator University of California, San Diego Ocean Observatories.

US IOOS ® Zdenka Willis Director, US IOOS. Programmatic year Awards Administration Transition.

IOOS National Glider Data Assembly Center

Eoin Howlett ASA, MARACOOS

Presentation transcript:

DMAC: Infrastructure to enable delivery of IOOS ® information Presentation to the IOOS Advisory Committee Derrick Snowden, System Architect, U.S. IOOS Program April 15, 2014

What is DMAC? DMAC is the collection of people, process, and technology that enable the dissemination of diverse and distributed data sets using the WWW as the platform. IOOS is deploying, and in some cases building, infrastructure to enable this dissemination

US IOOS Program Office DMAC efforts span three areas of focus. Technical solutions Image courtesy steve-dale.net Building community Implementation across the enterprise Image courtesy proactive IT Technical solutions

A layered view of the primary roles within IOOS/DMAC Technical Solutions4

5 Operational Use of HF Radar Surface Currents for Search And Rescue Surface Currents Search And Rescue Optimal Planning System (SAROPS) Data Acquisition Data Product Generation & Management

HFR SAROPS as an example of the layered architecture in practice. THREDDS Servers Technical solutions6

IOOS Model Data Interoperability Design Clients NetCDF -Java NetCDF4 -Python ERDDAP ROMS ADCIRC HYCOM SELFENCOM FVCOM Observed data (buoy, gauge, ADCP, glider) MatlabPanoplyIDVArcGISPythonGodiva2 Common Data Model Apply Standardization Distribute via Web Services OPeNDAP+CF WCS NetCDF Subset WMS ncISO SOS Grid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile Technical solutions Advances in a Distributed Approach for Ocean Model Data Interoperability; 7

Use Case: Model Data comparison using web services 33 Tb 4D model output on a triangular mesh grid CO-OPS offers 1019 data sets via their SOS containing water level and met parameters Plot the time series of observed water level and model simulation sampled at the observation locations, via web services Technical solutions8

Model Data comparison con’t com/rsignell- usgs/notebook/fef b49a db1ef3115e3 4d8271/CSW/IOOS_inundation.ipynb 9 Modify in your browser; Runs in the cloud; No need to download data;

Exposing Data! Implementation across the enterprise Functional DACs complement the regional footprint

Deployment view of the infrastructure available to regional associations to distribute in situ data Implementation across the enterprise11

Who does the program office work with most closely on DMAC technology issues? Building community12

Software supports the infrastructure: Reduce/reuse/recycle on github.com Building community13

US IOOS Program Office DMAC efforts span three areas of focus. Technical solutions Image courtesy steve-dale.net Building community Implementation across the enterprise Image courtesy proactive IT Technical solutions PeopleProcessTechnology