Rich Signell Roland Viger Curtis Price USGS Community for Data Integration Feb 15, 2012.

Slides:

Advertisements

Similar presentations

1 NASA CEOP Status & Demo CEOS WGISS-25 Sanya, China February 27, 2008 Yonsook Enloe.

Advertisements

Reading HDF family of formats via NetCDF-Java / CDM

The Model Output Interoperability Experiment in the Gulf of Maine: A Success Story Made Possible By CF, NcML, NetCDF-Java and THREDDS Rich Signell (USGS,

Integrating NOAA’s Unified Access Framework in GEOSS: Making Earth Observation data easier to access and use Matt Austin NOAA Technology Planning and Integration.

Streaming NetCDF John Caron July What does NetCDF do for you? Data Storage: machine-, OS-, compiler-independent Standard API (Application Programming.

Connecting HDF And ISO Metadata Ted Habermann, NASA/ESDIS Hook Hua, Barry Weiss, NASA/Jet Propulsion Lab Mike Folk, Gerd Heber, Elena Pourmal, The HDF.

McIDAS-V McIDAS-V The 5 th Generation of McIDAS by Tom Whittaker Space Science and Engineering Center University of Wisconsin-Madison USA with contributions.

Web based tools Ideas for presentation of operational meteorological data Ernst de Vreede KNMI EGOWS /6/2009 Ideas for presentation of operational.

Crossing the Digital Divide

2014 ESIP Summer Meeting July 8–11, 2014 | Frisco, Colorado Advancing Scientific Data Support in ArcGIS Nawajish Noman.

A Distributed Approach to Ocean, Atmosphere & Climate Model Data Interoperability Rich Signell USGS Woods Hole, MA COAWST Training Workshop Woods Hole,

Christine White, Esri Growing OPeNDAP Support: Current ArcGIS Workflows and Future Directions Christine White, Esri

Data Discovery 1.Data discovery systems 2.TDS and metadata 3.NcISO services.

ESRM 250 & CFR 520: Introduction to GIS © Phil Hurvitz, KEEP THIS TEXT BOX this slide includes some ESRI fonts. when you save this presentation,

OPeNDAP Present and Future An Overview Encompassing Current Projects & Potential New Directions Dave Fulker and James Gallagher.

Unidata TDS Workshop THREDDS Data Server Overview October 2014.

1 The NOAA Weather and Climate Toolkit Steve Ansari, Stephen Del Greco (NOAA / NCDC) Mark Phillips (UNC-Asheville / NEMAC) Bill Hankins (STG Inc.)

1 The NOAA Weather and Climate Toolkit Steve Ansari, Stephen Del Greco, Neal Lott (NOAA / NCDC)

OPeNDAP and the Data Access Protocol (DAP) Original version by Dave Fulker.

Quick Unidata Overview NetCDF Workshop 25 October 2012 Russ Rew.

Implementation of Model Data Interoperability for IOOS: Successes and Lessons Learned Rich Signell USGS Woods Hole, MA / NOAA Silver Spring USA Model Data.

Bringing it All Together: NODC’s Geoportal Server as an Integration Tool for Interoperable Data Services Kenneth S. Casey, Ph.D. YuanJie Li NOAA National.

Unidata’s TDS Workshop TDS Overview – Part II October 2012.

2 3 ROMS/COAWST NcML file 4 5 Exploiting IOOS: A Distributed, Standards-Based Framework and Software Stack for Searching, Accessing, Analyzing and.

Unidata TDS Workshop TDS Overview – Part I XX-XX October 2014.

Unidata’s Common Data Model John Caron Unidata/UCAR Nov 2006.

IOOS Data Management Integration Standards Plans in the Northeast Eric Bridger (GMRI) Sep

THREDDS Data Server Ethan Davis GEOSS Climate Workshop 23 September 2011.

1 The NOAA Weather and Climate Toolkit Steve Ansari, Stephen Del Greco, Neal Lott (NOAA / NCDC)

U.S. Department of the Interior U.S. Geological Survey Management of Oceanographic time-series data at the Woods Hole Coastal and Marine Science Center.

The IRI Climate Data Library: translating between data cultures Benno Blumenthal International Research Institute for Climate Prediction Columbia University.

NcML Aggregation vs Feature Collections. NcML functionality 1.Modify the objects found in CDM files – Especially Attributes – Don’t have to rewrite the.

Unidata’s TDS Workshop TDS Overview – Part II Unidata July 2011.

Mid-Course Review: NetCDF in the Current Proposal Period Russ Rew

Enhancements to a Community Toolset for Ocean Model Data Interoperability: Unstructured grids, NCTOOLBOX, and Distributed Search Rich Signell (USGS), Woods.

Tools in Support of a National DMAC Derrick Snowden NERACOOS/ODP Annual Meeting 26 Sep 2012.

Accomplishments and Remaining Challenges: THREDDS Data Server and Common Data Model Ethan Davis Unidata Policy Committee Meeting May 2011.

DMAC: Infrastructure to enable delivery of IOOS ® information Presentation to the IOOS Advisory Committee Derrick Snowden, System Architect, U.S. IOOS.

Unidata and Oceanography Through the Ages Rich Signell USGS Coastal and Marine Science Center Woods Hole, MA & NOAA Integrated Ocean Observing System (IOOS)

IOOS Model Data Interoperability Design ROMS POM WW3 WRF ECOM NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset THREDDS Data Server Standardized (CF)

Integrating Open Source Statistical Packages with ArcGIS Mark V. Janikas Liang-Huan (Leo) Chin.

GI-cat / THREDDS notes. GI-cat/THREDDS system

DAP4 James Gallagher & Ethan Davis OPeNDAP and Unidata.

Documentation from NcML to ISO Ted Habermann, NOAA NESDIS NGDC.

IOOS Modeling Testbed Cyberinfrastructure Rich Signell, USGS, Woods Hole, MA IOOS-RA-Briefing, Feb 14, 2012.

IOOS Coastal Ocean Modeling Testbed (COMT) Cyberinfrastructure Oceans 12 Becky Baltes, IOOS Liz Smith, SURA Rich Signell, USGS Eoin Howlett, Kyle Wilcox,

Unidata TDS Workshop THREDDS Data Server Overview

1 NASA CEOP Status & Demo CEOS WGISS-24 Oberpfaffenhofen, Germany October 15, 2007 Yonsook Enloe.

Unidata’s Common Data Model and the THREDDS Data Server John Caron Unidata/UCAR, Boulder CO Jan 6, 2006 ESIP Winter 2006.

IOOS Data Services with the THREDDS Data Server Rich Signell USGS, Woods Hole IOOS DMAC Workshop Silver Spring Sep 10, 2013 Rich Signell USGS, Woods Hole.

THREDDS Catalogs Ethan Davis UCAR/Unidata NASA ESDSWG Standards Process Group meeting, 17 July 2007.

Unidata’s TDS Workshop TDS Overview – Part I July 2011.

Improving Data Catalogs with Free and Open Source Software Kevin O’Brien University of Washington Joint Institute for the Study of the Atmosphere and Ocean.

GI-cat / THREDDS notes. GI-cat/THREDDS system

The Unified Access Framework for Gridded Data … the 1 st year focus of NOAA’s Global Earth Observation Integrated Data Environment (GEO-IDE) Steve Hankin,

Data Interoperability at the IRI: translating between data cultures Benno Blumenthal International Research Institute for Climate Prediction Columbia University.

U.S. INTEGRATED OCEAN OBSERVING SYSTEM DATA ACCESS SERVICES Coastal Geotools: Advances in Federal Data Access Services Hassan Moustahfid, Chris Duncombe.

DOC / NOAA / NMFS / SWFSC / ERD

Data Stewardship at the NOAA Data Centers Sub Topic - Value Added Products ESIP Federation Meeting, Washington, DC January 6-8, 2009.

Catalog-driven workflows using CSW Rich Signell, USGS, Woods Hole, MA, USA Filipe Fernandes, SECOORA, Brazil Kyle Wilcox, Axiom Data Science, Wickford,

ORNL DAAC SPATIAL DATA ACCESS TOOL Open Geospatial Consortium (OGC) Services Bruce E. Wilson Suresh K. Santhana Vannan Yaxing Wei Tammy W. Beaty National.

OGC Web Services with complex data Stephen Pascoe How OGC Web Services relate to GML Application Schema.

Enabling the Transition of CPC Products to GIS Format Brian Doty Jennifer Adams Michael Halpert Viviane Silva.

Update on Unidata Technologies for Data Access Russ Rew

GeoServer Prof. Wenwen Li School of Geographical Sciences and Urban Planning 5644 Coor Hall

Unidata Infrastructure for Data Services Russ Rew GO-ESSP Workshop, LLNL

IRI/LDEO Climate Data Library M.Benno Blumenthal, Michael Bell, and John del Corral International Research Institute for Climate and Society Columbia University.

Data Are from Mars, Tools Are from Venus

CEE 6440 GIS in Water Resources Fall 2004 Term Paper Presentation

HDF-EOS Workshop XXI / The 2018 ESIP Summer Meeting

Presentation transcript:

Rich Signell Roland Viger Curtis Price USGS Community for Data Integration Feb 15, 2012

 Web Mapping Services  Web Feature Services  Web Coverage Services  OPeNDAP  All use HTTP  Subsetting  Resampling  Different data types, data models

 ESRI ArcGIS/SDE  ESRI Grid  WMS, WCS (WFS)  THREDDS (many other options)  NetCDF, CF, HDF, GRIB, …  WMS, WCS, OPeNDAP

 Python  ArcGIS w/Python  MATLAB  …

 Traditional raster  Structured curvilinear  TINs

 Scipy, numpy  come w/ArcGIS  Enthought  Packages lots of great python packages together  Not always readily usable w/an ArcGIS install

Uses 10.1, LandXML

Reads UGRID format into ESRI Shapefile, approximates a TIN (w/shapefile of polygons & shapefile of points)

Enthought maintains this code.

Tons of great functionality beyond ArcGIS!

 ESRI and Enthought ensure distributions are compatible  ArcGIS leverage the 100 packages in the Enthought Python Distribution  Serious Python users to leverage ESRI tools

 Unidata has a Java Common Data Model (CDM)  C Common Data Model (libCF) still in development  We need a Python (or C) CDM library!  When we have one, ESRI can leverage it!

 ArcGIS more of an integrative platform  ArcGIS users can exploit the world of OPeNDAP data  Dan Sampson, coastal zone manager in Massachusetts, working in ArcGIS, can create his own products from OPeNDAP Data

ROMS POM WW3 WRF ECOM NcML Common Data Model OPeNDAP+CF WCS NetCDF Subset Service THREDDS Data Server Standardized (CF) Virtual Datasets Nonstandard Output Files (distributed) Web Services Matlab Panoply IDV ERDDAP Clients NetCDF -Java Library or Service NcML, NetCDF-Java and THREDDS Data Server built and supported by Unidata under NSF-support WMS ncISO ArcGIS10 NetCDF4- Python FVCOM Python EDC ERDDAP NetCDF-Java

Geoportal Server Broker Data ServersCatalog Services THREDDS-ncISO OGC-WCS OpenSearch OGC-CSW THREDDS-ncISO Clients GI-CAT Geoportal Server GI-Portal Geonetwork Python Matlab OGC-SOS

NCTOOLBOX links = opensearch(q) nc = ncugrid(links.dap{1}) z = nc.data(‘zeta’,…) grid = nc.grid(‘zeta’,…) z = 26441x1 single grid = lat: [26441x1 single] lon: [26441x1 single] time: (matlab datenum) connectivity: [52025x3 int32]

 ArcGIS10.0 has Python integration  There are several Python OPeNDAP modules  How about PyDAP? NO!  NetCDF4-Python? YES! Common interface for NetCDF3, NetCDF4 files and OPeNDAP datasets! (Uses Unidata’s NetCDF C library w/native OPeNDAP)  But wait… NetCDF4-Python binary had not been built for Windows  Python community’s Christoph Gohlke (UCI) to the rescue!!! (Python 2.6/2.7/3.0 Win32/Win64 and special ArcGIS10.0 and ArcGIS10.1 builds)

 Create Mosaic Dataset  Add Raster to Mosaic Dataset  Open Attribute Table  Add time field  Edit Mosaic Dataset  Add date value to each raster layer  Enable time on Mosaic via LayerProperties  Use Time Slider to play around!