ESIP Air Quality Jan 2006 1 Air Quality Cluster Air Quality Cluster Technology Track Earth Science Information Partners Partners NASA NOAA EPA (?) USGS.

Slides:

Advertisements

Similar presentations

WCS Data Exchange at the DataFed Server/Client Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University, St. Louis, MO OGC TC.

Advertisements

The Next Generation Network Enabled Weather (NNEW) SWIM Application Asia/Pacific AMHS/SWIM Workshop Chaing Mai, Thailand March 5-7, 2012 Tom McParland,

Web Services Implementation Case Study: DataFed Air Quality Data & Services Project Coordinators: Software Architecture: R. Husar Software Implementation:

REASoN REASoN Project to link NASA's data, modeling and systems to users in research, education and applications Application of NASA ESE Data and Tools.

OGC Demo at IGARSS06 July 30 - Denver, CO Telecon 11 July 2006 Liping Di, George Mason University Rudolf Husar, Washington University.

Virtual Product of Air Quality Demo Scripts Liping Di, Peisheng Zhao Center for Spatial Information Science and Systems (CSISS) George Mason University.

Web 2.0 in Air Quality Analysis and Management Rudolf B. Husar and Erin M. Robinson Washington University, St. Louis, MO 2007 National Environmental Information.

ESIP Vision: “Achieve a sustainable world” by Serving as facilitator and advisor for the Earth science information community Promoting efficient flow of.

Planned Title: Review of Evaluation of Geospatial Search Allan Doyle.

Stefan Falke Center for Air Pollution Impact and Trend Analysis Washington University in St. Louis Networked Data and Tools for Environmental Management.

Distributed Data Analysis & Dissemination System (D-DADS) Prepared by Stefan Falke Rudolf Husar Bret Schichtel June 2000.

OPeNDAP and the Data Access Protocol (DAP) Original version by Dave Fulker.

AIRNow-International The future of the United States real-time air quality reporting and forecasting program and GEOSS participation John E. White U.S.

Distributed Voyager (DVoy) Web Services

DRAFT June 6, 2005 ESIP AQ Cluster, Air Quality Cluster Air Quality Cluster TechTrack Earth Science Information Partners Partners NASA.

Unidata’s TDS Workshop TDS Overview – Part II October 2012.

Instrument Builders Information Specialists (ESIP) Scientists Curriculum Developers Teachers Decision Analysts Decision Makers Reports From Kim Kastens.

Data Scrounging 101 Steve Signell, Instructor Robert Poirier, TA School of Science Rensselaer Polytechnic Institute Monday,

REASoN REASoN Project to link NASA's data, modeling and systems to users in research, applications and education Application of NASA ESE Data and Tools.

Architecture and Technologies for an Agile, User-Oriented Air Quality Data System Rudolf B. Husar Washington University, St. Louis Presented at the workshop.

WCS Data Exchange at the DataFed Server/Client Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University, St. Louis, MO OGC TC.

Federated Network for Sharing Air Quality Data and Processing Services Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University,

1 International Standards for Data Interoperability GALEON Geo-interface for Air, Environment, Land, Ocean NetCDF Ben Domenico Unidata Program Center*

Work Group Meeting on HTAP-Relevant IT Techniques, Tools and Philosophies: DataFed Experience and Perspectives Rudolf B. Husar CAPITA, Washington University,

REASoN REASoN Project to link NASA's data, modeling and systems to users in research, education and applications Application of NASA ESE Data and Tools.

1 NASA CEOP Status & Demo CEOS WGISS-24 Oberpfaffenhofen, Germany October 15, 2007 Yonsook Enloe.

Spatio-Temporal Data Sharing using XML Web Services Presented at the Workgroup Meeting on Web-based Environmental Information System for Global Emission.

Current Air Quality Information ‘Ecosystem’ (Draft for Feedback) AQ information includes emissions, ambient & satellite data and model outputs The distributed.

Stefan Falke Center for Air Pollution Impact and Trend Analysis Washington University in St. Louis Brooke Hemming US EPA – Office of Research and Development.

Accessing and Using Fire-Related Data with the CAPITA DataFed.net* Services Framework Stefan Falke Rudolf Husar Kari Hoijarvi Washington University in.

Air Quality Data Services: Application of OGC specifications Air Quality Data: Multi-dimensional, multi-source, multi-format Point observations are collected.

Unidata’s TDS Workshop TDS Overview – Part I July 2011.

NASA Air Quality Applications Program and the ESIP Air Quality Cluster The goal of the NASA Air Quality Management program is to: Enable partners’ beneficial.

OGC Demo at IGARSS06 July 30 - Denver, CO Telecon 11 July 2006 Liping Di, George Mason University Rudolf Husar, Washington University.

Stefan Falke and Rudolf Husar Center for Air Pollution Impact and Trend Analysis Washington University in St. Louis A NSF Digital Government Pilot Project.

ESIP Vision: “Achieve a sustainable world” by Serving as facilitator and advisor for the Earth science information community Promoting efficient flow of.

Enabling Tools and Methods for International, Inter- disciplinary and Educational Collaboration E. M. Robinson, K. Hoijarvi, S. Falke, E. Fialowski, M.

Interoperability of Data and Processing Services: Experience Using the DataFed System Rudolf B. Husar, Stefan R. Falke and Kari Hoijarvi Washington University,

Interoperability of Data and Processing Services: Experience Using the DataFed System Rudolf B. Husar, Stefan R. Falke and Kari Hoijarvi Washington University,

Interoperability = Leverage + Collaboration  Chris Lynnes  GES DISC.

Federated Space-Time Query for Earth Science Data Using OpenSearch Conventions ESIP Federated Search Cluster Chris Lynnes Bruce Beaumont Ruth Duerr Hook.

Part I DataFed An Agile Distributed Air Quality Data System Rudolf B. Husar Washington University, St. Louis Seminar Presented at University of Alabama,

Distributed Data Analysis & Dissemination System (D-DADS ) Special Interest Group on Data Integration June 2000.

Fire Emissions Network Sept. 4, 2002 A white paper for the development of a NSF Digital Government Program proposal Stefan Falke Washington University.

NASA REASoN Project SHAirED: S ervices for H elping the Air -quality Community use E SE D ata Stefan Falke, Kari Höijärvi and Rudolf Husar, Washington.

NASA REASoN Project SHAirED: S ervices for H elping the Air -quality Community use E SE D ata Stefan Falke, Kari Höijärvi and Rudolf Husar, Washington.

Grids and Beyond: netCDF-CF and ISO/OGC Features and Coverages Ethan Davis, John Caron, Ben Domenico UCAR/Unidata AMS IIPS, 23 January 2008.

Part I DataFed An Agile Distributed Air Quality Data System Rudolf B. Husar Washington University, St. Louis Seminar Presented at University of Alabama,

Data Stewardship at the NOAA Data Centers Sub Topic - Value Added Products ESIP Federation Meeting, Washington, DC January 6-8, 2009.

Web Services-Based Mediator of Distributed Data Flow and Processing Project Coordinators: Software Architecture: R. Husar Software Implementation: K. Höijärvi.

ORNL DAAC SPATIAL DATA ACCESS TOOL Open Geospatial Consortium (OGC) Services Bruce E. Wilson Suresh K. Santhana Vannan Yaxing Wei Tammy W. Beaty National.

Application of NASA ESE Data and Tools to Air Quality Management Stefan Falke and Rudolf Husar (Co-PIs) Washington University in St. Louis Project Period:

: Data Sharing/Processing Infrastructure Data Catalog and Access Dozens of datasets on aerosols, emissions, fire, meteorology,

The Federated Data System DataFed: Experiences in Data Homogenization and Networking R.B. Husar, K. Hoijarvi, S. R. Falke, E. M. Robinson, Washington University,

1 Key Results from GALEON John Caron Ben Dominico UCAR/Unidata.

DRAFT June 6, 2005 ESIP AQ Cluster, Contact R. Husar Air Quality Cluster Air Quality Cluster TechTrack Earth Science Information Partners.

Application of NASA ESE Data and Tools to Particulate Air Quality Management A proposal to NASA Earth Science REASoN Solicitation CAN-02-OES-01 REASoN:

1 2.5 DISTRIBUTED DATA INTEGRATION WTF-CEOP (WGISS Test Facility for CEOP) May 2007 Yonsook Enloe (NASA/SGT) Chris Lynnes (NASA)

The Federated Data System DataFed R. Husar, K. Hoijarvi, S. Falke, DaFed Community EPA Data Summit, Feb. 12, 2008, RTP Non-intrusive data integration infrastructure.

Standards-based Access to Air Quality Data: Application of OGC WMS and WCS Protocols Client Server Std. Interface GetCapabilities GetData Capabilities,

Voyager Data Services Services for Finding, Exploring and Presenting Distributed Environmental Data Outline Prepared by Voyager Interest Group on Environmental.

Federated Network for Sharing Air Quality Data and Processing Services Center for Air Pollution Impact and Trend Analysis (CAPITA) Washington University,

Interoperability Day Introduction Standards-based Web Services Interfaces to Existing Atmospheric/Oceanographic Data Systems Ben Domenico Unidata Program.

Serving society Stimulating innovation Supporting legislation Web Coverage Services (WCS) Thematic Cluster #3 Jordi Escriu Facilitator.

Fire, Smoke & Air Quality: Tools for Data Exploration & Analysis : Data Sharing/Processing Infrastructure This project integrates.

OGC Live Demo at GEOSS Workshop July 30 - Denver, CO Demo Scripts Liping Di, Peisheng Zhao, George Mason University Rudolf Husar,

NATIONAL AERONAUTICS AND SPACE ADMINISTRATION ESDS Reuse Working Group Earth Science Data Systems Reuse Working Group Case Study: SHAirED Services for.

ESIP Air Quality Jan Air Quality Cluster Air Quality Cluster Technology Track Earth Science Information Partners Partners NASA NOAA EPA (?) USGS.

The Next Generation Network Enabled Weather (NNEW) SWIM Application

GEOSS Air Quality Community Infrastructure

Presentation transcript:

ESIP Air Quality Jan Air Quality Cluster Air Quality Cluster Technology Track Earth Science Information Partners Partners NASA NOAA EPA (?) USGS DOE NSF Industry… Data Flow & Interoperability in DataFed Service-based AQ Analysis System R. B. Husar, S. R. Falke and K. Höijärvi Washington University, St. Louis, MO ESIP Federation Winter Meeting 2006 Washington, DC, January 4, 2006

ESIP Air Quality Jan DataFed in a Nutshell DataFed Vision Aid air quality analysis by effective use of relevant data DataFed Goals Facilitate access and flow of AQ data from provider to users Support user-driven data processing value chains Participate in specific application projects Approach: Mediation Between Users and Data Providers DataFed assumes spontaneous, autonomous data providers Non-intrusively wraps datasets for access by web services Mediates, homogenizes data views. e.g. geo-spatial, time... Applications Browsers and analysis tools for distributed monitoring data Serve as data gateway for user programs; GIS, science tools DataFed is focused on the mediation of air quality data

ESIP Air Quality Jan Typical DataFed AQ Analysis Tools ConsolesConsoles: Data from diverse sources are displayed to create a rich context for exploration and analysis CATTCATT: Combined Aerosol Trajectory Tool for the browsing backtrajectories for specified chemical conditions Tool Viewer: Viewer: General purpose spatio-temporal data browser and view editor applicable for all DataFed datasets

ESIP Air Quality Jan Web Services: Building Blocks of DataFed Programming Access, Process, Render Data by Service Chaining [Station] [Monitor] Clickable monitor locations layer Map boundary layer

ESIP Air Quality Jan Single Data Model for All AQ Data Most Views are slices through a cube of data organized by lat, lon, altitude, and time (X,Y,Z,T) Multidimensional Data Cube

ESIP Air Quality Jan OGC Abstract Specification – Coverage (00-106)Coverage (00-106) Abstract Spec.: Coverages model and visualize spatial relationships between, and the spatial distribution of, earth phenomena Coverage subtypes of Air Quality interest are Image, Grid and Discrete Point Has a property (Coverage_Function) that has a spatial domain, and a value set. For Image and Grid, the Coverage_Function is an array…Discrete Point coverage is described by a PointC_Function where the spatial domain consists of a collection of points Observation: The Abstract Spec. foresaw the need for Point Coverage, i.e. fixed monitoring Stations

ESIP Air Quality Jan OGC Web Coverage Service (WCS) Specification HTTP GET/POST based interfaces Services have XML service descriptions (“Capabilities”, “Description”) Filter parameters allow selection of subsets of source data Output formats advertised by each service instance OGC WCS getCoverage Schema Suitable for wrapping with SOAP envelope, WSDL access, loose coupling WCS is for "coverages" – information representing space-time-varying phenomena WCS describes, requests and delivers coverages in spatio-temporal domain WCS version 1.1 is limited to grids/"simple” coverages with homogeneous range sets

ESIP Air Quality Jan through Data Access through Adapters DataFed SOAP,HTTP Get OGC WCS HTTP Get, Post OGC WMS HTTP Get Station-Point SQL Server, Files… Sequence Image, file nDim Grid OpenDAP NetCDF, … Other Traject., Event, Pic Sources Diverse formats Many data models Data Wrapper Data into geo-cubes Queries to views Virtual Data Cube Global geo-cube data model Makes queries data-neutral Others? e.g. OpenDAP Output Protocol dependent User specified GeoTable CSV,XLS,GML GeoGrid GML,NetCDF.. GeoImage GeoTIFF, PNG.. Other MS Dataset.. Query Adapter Maps query to protocol User selects protocols

ESIP Air Quality Jan OGC WCS Demonstration: AirNOW Station DatasetAirNOW Map View Services WCS Query s&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EPSG:432 6&COVERAGE=AIRNOW.pmfine&FORMAT=CSV&BBOX= , , , ,0,0&TIME= T15:00:00Z&WIDTH=999&HEIGHT=999&DEPTH=999 Formats: CSV, NetCDF-table, etc Time View Services =wcs&REQUEST=GetCoverage&VERSION=1.0.0&CRS=EP SG:4326&COVERAGE=AIRNOW.pmfine&FORMAT=CSV&B BOX= , , , ,0,0&TIME= T00:00:00Z/ T00:00:00Z&WIDTH=999&HEIGHT=999&DEPTH=999 Formats: CSV, NetCDF-table, etc

ESIP Air Quality Jan Data Access Services 1. Click Services Button 4. Set Output Format 3. Access Settings 2. Select Data Access Protocol 4. Execute WCS Query in Browser or in Program 5. Consume data by program, service or viewer

ESIP Air Quality Jan GALEON Interoperability Experiment GALEON Geo-interface for Atmosphere, Land, Earth, and Ocean netCDF Lead by UNIDATA & OGCUNIDATAOGC Participants: academia, industry, international Unify Earth Science & GIS Data Flows B. Domenico GALEON UNIDATA

ESIP Air Quality Jan GALEON: Earth Science – GIS Interoperability Shared Data Model

ESIP Air Quality Jan OGC WCS Demonstration: THREDDS_GFS 4Dim DatasetTHREDDS_GFS Lat/Lon Box Elev Range Time Range Map: BBOX=-180,-90,180,90, 1350,1350& TIME= / /PT3H Time: BBOX=-34,49.05,-34,49.05, 1350,1350& TIME= / /PT3H Elev: BBOX=-34,49.05,-34,49.05, 0,18000 & TIME= / /PT3H The form of the WCS query is the same for all slices through the data cube (views) The only difference in the views is the thickness of the slices in each dimension Return grid is in multiple formats (NetCDF, CSV, GML, PNG, … ) Map View Services WCS Query Time View Services WCS Query Elevation View Services WCS Query WCS Query

ESIP Air Quality Jan OGC WCS Demonstration: Grid, Image, Station Data Types Coverage=THEEDDS.T& BBOX=-126,24,-65,52,0,0 &TIME= / &FORMAT=NetCDF Coverage=SURF.Bext& BBOX=-126,24,-65,52,0,0 &TIME= / &FORMAT=NetCDF-table Coverage=SEAW.Refl& BBOX=-126,24,-65,52,0,0 &TIME= / &FORMAT=GeoTIFF COVERAGE=sst& BBOX=-126,24,-65,52,0,0 &TIME= , &FORMAT=NetCDF UNIDATA – THREDDS/GALEON WCS THREDDS/GALEON WCS DataFed GALEON WCS U Florence, It GALEON WCS DataFed GALEON WCS Grid Image Station Services WCS Query Services WCS Query Services WCS Query Services WCS Query

ESIP Air Quality Jan Benefits of WMS Compatibility: Geospatial One-Stop Provides Access to DataFed spatial layers through WMS Problem: No Time Dim! DataFed/VIEWS Asian Dust Concentration (circles) S. Falke

ESIP Air Quality Jan Summary OGC WCS protocol is suitable for delivering most air quality data: Suitable for most data view queries BBOX, TIME Station-Point SQL Server, Files… Sequence Image, file nDim Grid OpenDAP NetCDF, … Applicable to most data types Station, Image, Grid For Air Quality data needs: WCS: Add Point coverage to WCS (in addition to grid, polygon) WCS: Strongly typed (XML Schema-verifiable) return data types Testing: Much fixing and testing for server-client compatibility