SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Hydrologic Information System Services Architecture Collaborative project: UTAustin (D. R. Maidment) + SDSC.

Slides:

Advertisements

Similar presentations

Some notes on CyberGIS in hydrology Ilya Zaslavsky Spatial Information Systems Lab San Diego Supercomputer Center UCSD TeraGrid CyberGIS Workshop, February.

Advertisements

HydroServer A Platform for Publishing Space- Time Hydrologic Datasets Support EAR CUAHSI HIS Sharing hydrologic data Jeffery.

Sharing Hydrologic Data with the CUAHSI Hydrologic Information System Support EAR CUAHSI HIS Sharing hydrologic data David.

ICEWATER: INRA Constellation of Experimental Watersheds Cyberinfrastructure to Support Publication of Water Resources Data Jeffery S. Horsburgh, Utah State.

Linking HIS and GIS How to support the objective, transparent and robust calculation and publication of SWSI? Jeffery S. Horsburgh CUAHSI HIS Sharing hydrologic.

This work is funded by National Science Foundation Grant EAR Accessing and Sharing Data Using the CUAHSI Hydrologic Information System CUAHSI HIS.

CUAHSI HIS Data Services Project David R. Maidment Director, Center for Research in Water Resources University of Texas at Austin (HIS Project Leader)

Services-Oriented Architecture Updates David Maidment Part of a presentation made to the HIS Standing Committee, Washington DC, Nov 15, 2010.

Components of an Integrated Environmental Observatory Information System Cyberinfrastructure to Support Publication of Water Resources Data Jeffery S.

This work was funded by the U.S. National Science Foundation under grant EAR Any opinions, findings and conclusions or recommendations expressed.

HydroServer A Platform for Publishing Space- Time Hydrologic Datasets Support EAR CUAHSI HIS Sharing hydrologic data Jeffery.

Time Series Analyst An Internet Based Application for Viewing and Analyzing Environmental Time Series Jeffery S. Horsburgh Utah State University David.

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D A SCALABLE SYSTEM FOR ONLINE ACCESS TO NATIONAL AND LOCAL REPOSITORIES OF HYDROLOGIC TIME SERIES Ilya Zaslavsky,

Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.

Two NSF Data Services Projects Rick Hooper, President Consortium of Universities for the Advancement of Hydrologic Science, Inc.

Using GIS in Creating an End-to- End System for Publishing Environmental Observations Data Jeffery S. Horsburgh David G. Tarboton, David R. Maidment, Ilya.

Integrating Historical and Realtime Monitoring Data into an Internet Based Watershed Information System for the Bear River Basin Jeff Horsburgh David Stevens,

Deployment and Evaluation of an Observations Data Model Jeffery S Horsburgh David G Tarboton Ilya Zaslavsky David R. Maidment David Valentine

CUAHSI Hydrologic Information Systems and Web Services By David R. Maidment With support from many collaborators: Ilya Zaslavsky, David Valentine, Reza.

HIS Team and Collaborators University of Texas at Austin – David Maidment, Tim Whiteaker, Ernest To, Bryan Enslein, Kate Marney San Diego Supercomputer.

GIS at SDSC Domains: –From geology, environmental science, hydrology, ocean biodiversity, regional development, Katrina response, archaeology, to neuroscience.

SAN DIEGO SUPERCOMPUTER CENTER Developing a CUAHSI HIS Data Node, as part of Cyberinfrastructure for the Hydrologic Sciences David Valentine Ilya Zaslavsky.

An End-to-End System for Publishing Environmental Observations Data Jeffery S. Horsburgh David K. Stevens, David G. Tarboton, Nancy O. Mesner, Amber Spackman.

A Services Oriented Architecture for Water Resources Data David R. Maidment Center for Research in Water Resources University of Texas at Austin EPA Storet.

Over-allocation to irrigation Bushfire recovery impacts Expanding plantations Drying and warming climate Uncapped groundwater extraction Expanding farm.

A Services Oriented Architecture for Water Resources Data David R. Maidment Center for Research in Water Resources University of Texas at Austin.

About CUAHSI The Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) is an organization representing 120+ universities.

Ocean Sciences What is CUAHSI? CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001 as a legal entity Program.

Hydrologic Information System for the Nation Ilya Zaslavsky Spatial Information Systems Lab San Diego Supercomputer Center UCSD UCSD Seminar, February.

About CUAHSI The Consortium of Universities for the Advancement of Hydrologic Science, Inc. (CUAHSI) is an organization representing 120+ universities.

Water Web Services David R. Maidment Center for Research in Water Resources University of Texas at Austin Open Waters Symposium Delft, the Netherlands.

Hydrologic Information System for the Nation I. Zaslavsky (SDSC) & The CUAHSI HIS Project his.cuahsi.org, hiscentral.cuahsi.org.

Data Interoperability in the Hydrologic Sciences The CUAHSI Hydrologic Information System David Tarboton, David Maidment, Ilya Zaslavsky, Dan Ames, Jon.

CUAHSI Hydrologic Information System an introduction Ilya Zaslavsky Director, Spatial Information Systems Lab San Diego Supercomputer Center University.

Advancing an Information Model for Environmental Observations Jeffery S. Horsburgh Anthony Aufdenkampe, Richard P. Hooper, Kerstin Lehnert, Kim Schreuders,

Publishing Observations Data: from ODM to HIS Central.

Hydrologic Information System for the Nation David R. Maidment Director Center for Research in Water Resources, University of Texas at Austin.

Hydrologic Information System for the Nation Ilya Zaslavsky Spatial Information Systems Lab San Diego Supercomputer Center UCSD UCSD Seminar, February.

Hydrologic Information System for the Nation Ilya Zaslavsky Spatial Information Systems Lab San Diego Supercomputer Center UCSD BOM talk, Melbourne, March.

Hydrologic Information System for the Nation I. Zaslavsky (SDSC) & The CUAHSI HIS Project his.cuahsi.org, hiscentral.cuahsi.org.

CUAHSI, WATERS and HIS by Richard P. Hooper, David G. Tarboton and David R. Maidment.

CUAHSI Hydrologic Information Systems. HIS Project Team Yao Liang John Helly Project co-PI Collaborator.

Web Services and Water Markup Language for Distributed Hydrologic Data Access Ilya Zaslavsky San Diego Supercomputer Center, UCSD CUAHSI = Consortium of.

Overview of CUAHSI HIS Version 1.1 David R. Maidment Director, Center for Research in Water Resources University of Texas at Austin CUAHSI Biennial Science.

Hydrologic Information System for the Nation I. Zaslavsky (SDSC) & The CUAHSI HIS Project his.cuahsi.org, hiscentral.cuahsi.org.

CBEO:N Chesapeake Bay Environmental Observatory as a Network Node About CBEO The mission of the CBEO project is development of a Chesapeake Bay Environmental.

WEB SERVICES FOR UNIFIED ACCESS TO NATIONAL HYDROLOGIC DATA REPOSITORIES AND REAL TIME OBSERVATION DATA: CUAHSI HIS EXPERIENCE Ilya Zaslavsky San Diego.

CUAHSI Hydrologic Information Systems David R. Maidment Center for Research in Water Resources University of Texas at Austin and Ilya Zaslavsky, David.

The CUAHSI Community Hydrologic Information System Jeffery S. Horsburgh Utah Water Research Laboratory Utah State University CUAHSI HIS Sharing hydrologic.

Bringing Water Data Together David R. Maidment Center for Research in Water Resources University of Texas at Austin Texas Water Summit San Antonio Tx,

Hydrologic Modeling in 2011 David R. Maidment Center for Research in Water Resources University of Texas at Austin Leader of the CUAHSI Hydrologic Information.

Hydrologic Information System GIS – the water environment Water Resources – the water itself CUAHSI HIS: NSF-supported collaborative project: UT Austin.

CUAHSI HIS Features of Observations Data Model. NWIS ArcGIS Excel NCAR Trends NAWQA Storet NCDC Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++

GEON2 and OpenEarth Framework (OEF) Bradley Wallet School of Geology and Geophysics, University of Oklahoma

CUAHSI Hydrologic Information Systems David R. Maidment and Ernest To Center for Research in Water Resources, University of Texas at Austin Hydrosystems.

Sharing SRP Water Sample Data Using CUAHSI HIS Infrastructure Ilya Zaslavsky, Thomas Whitenack, Keith Pezzoli, Hiram Sarabia University of California at.

The CUAHSI Observations Data Model Jeff Horsburgh David Maidment, David Tarboton, Ilya Zaslavsky, Michael Piasecki, Jon Goodall, David Valentine,

CUAHSI HIS: Science Challenges Linking small integrated research sites (

Ocean Sciences CUAHSI-Hydrologic Information Systems CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001.

1 CUAHSI Web Services and Hydrologic Information Systems By David R. Maidment, University of Texas at Austin Collaborators: Ilya Zaslavsky and Reza Wahadj,

Hydroinformatics Lecture 15: HydroServer and HydroServer Lite The CUAHSI HIS is Supported by NSF Grant# EAR CUAHSI HIS Sharing hydrologic data.

Developing a community hydrologic information system David G Tarboton David R. Maidment (PI) Ilya Zaslavsky Michael Piasecki Jon Goodall

The CUAHSI Hydrologic Information System Spatial Data Publication Platform David Tarboton, Jeff Horsburgh, David Maidment, Dan Ames, Jon Goodall, Richard.

Using GIS in Creating an End-to-End System for Publishing Environmental Observations Data Jeffery S. Horsburgh David G. Tarboton, David R. Maidment, Ilya.

Sharing Hydrologic Data with the CUAHSI* Hydrologic Information System

The CUAHSI Hydrologic Information System and NHD Plus A Services Oriented Architecture for Water Resources Data David G Tarboton David R. Maidment (PI)

The CUAHSI Hydrologic Information System Service Oriented Architecture for Water Resources CUAHSI HIS Sharing hydrologic data Support.

Lecture 8 Database Implementation

CUAHSI HIS Sharing hydrologic data

HydroDesktop: A Key Component of the CUAHSI/CZO HIS for Hydrologic Data Discovery, Visualization, and Analysis Daniel P. Ames, Ph.D. P.E. Idaho State University.

Presentation transcript:

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Hydrologic Information System Services Architecture Collaborative project: UTAustin (D. R. Maidment) + SDSC (C. Baru, I. Zaslavsky) + Utah State U (D. Tarboton)+ Drexel U (M. Piasecki) + Duke U. (J. Goodall) + CUAHSI Office (R. Hooper)

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D The Grid is becoming the backbone for collaborative science and data sharing CI is about RE-USING data and research resources !!

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D CI Vision for Hydrologic Science Leverage ongoing cyberinfrastructure projects: The Geosciences Network (GEON) Share data between Earth Disciplines Secure access to Grid resources, single sign-on authentication/ authorization, distributed data management, data publication, search, information integration, knowledge management, scientific workflows, archiving Integrate with common COTS (commercial off-the shelf) software: Excel, ArcGIS, Matlab… and Fortran … mostly on Windows… Interesting survey of CUAHSI partners by David Tarboton!

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D HIS User Assessment (Chapter 4 in Status Report) Data Access Science Observatory support Education Which of the four HIS goals is most important to you?

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Tuning to unique features of hydrology Hydrologic observations: Reliance on federally-organized data collection (NWIS, STORET, Ameriflux, etc.) with huge and complex nomenclatures  simplifying access to federal repositories  relatively lower emphasis on data ownership Handling time in both UTC and local Various spatial offsets Multiple data types: time series, fields, spatial data Integrative discipline: Interoperation with atmospheric, ocean, soils, geomorphology, social datasets and services… Community: Organized by “natural boundaries”  natural object hierarchy  networks of relatively autonomous self-managed data nodes Partnership with public sector water management

WaterOneFlow Web Services Data accessthrough web services Data storage through web services Downloads Uploads Observatory servers SDSC HIS servers 3 rd party servers e.g. USGS, NCDC GIS Matlab IDL Splus, R D2K, I2K Programming (Fortran, C, VB) Web services interface Web portal Interface (HDAS) Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses HTML -XML WSDL - SOAP Hydrologic Information System Service Oriented Architecture

Main Components Web services for accessing hydrologic repositories Hydrologic Observations Data Model Hydrologic Data Access System + Time Series Viewer Collection of CUAHSI nodes NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++ CUAHSI Web Services

Database Sizes EPA NWS USGS Records 200 million ? Stations Time range 250 million 800, years 1.5 million 100 years 19,000 (From Jon Goodall, Duke U.)

Language for Data Representation EPA NWS USGS Unique Identifier for a Observation Station site_no Station ID COOPID Latitude, Longitude Time of Measurement Station Latitude, Station Longitude Activity Start dec_lat_va, dec_long_va dv_dt YEAR,MO,DA,TIME LATITUDE, LONGITUDE Lots of semantic differences in parameter names, methods, etc.

CUAHSI Search Wizard Inside the NWIS Module Search Wizard Demo NWIS Parameter Codes Output (Drexel Univ.) Ontology Viz Demo

WaterOneFlow Web Services ServiceInputOutput GetSites Obs Network, filter Get station codes in network GetSiteInfo Station CodeLat/long, station name GetVariables Obs Network or data source, filter Get variable codes GetVariableInfo Variable codeDescription of variable GetValues Station code or lat/long point, variable code, begin date, end date A time series of values GetChart As for GetValueA chart plotting the values Output: string standardized to a common set of objects (XML schema)

CUAHSI Web Services NCEP North American Forecast Model 12 Km grid for continental US New services:

CUAHSI Point Hydrologic Observations Data Model A relational database stored in Access, PostgreSQL, MS SQL Server, …. Stores observation data made at points Consistent format for storage of observations from many different sources and of many different types. Streamflow Flux tower data Precipitation & Climate Groundwater levels Water Quality Soil moisture data (D. Tarboton, USU) Community design requirements (22 reviewers)

Schema

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Uses and tools for HODM HODM is central to HIS infrastructure, but lacks tools Testing HODM with two types of data: federal repositories, and external databases (Panola). Personal and enterprise versions. Mapping wizard: loading Excel observation data to HODM database: Can save mapping files for subsequent runs of similarly formatted spreadsheets Local data analysis can be done: charts and stats HDAS as an interface to HODM datasets - but shall not be the only one - so exposing HODM as Web services

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Hydrologic Data Access System

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Hydrologic Data Access System

Cross-platform design Central CUAHSI HIS Node (Windows) GEON Data Node (Linux) Data Apache Tomcat IIS Web Server ASP.Net Geon Software Stack SQL Server Proxy ArcGIS Technologies HDAS HODM Web Service Web Services Web Service proxies Data Remote CUAHSI HIS Node (Windows) Data IIS Web Server ASP.Net SQL Server ArcGIS Technologies HDAS HODM Web Service Web Services Web Service proxies Remote CUAHSI HIS Node (Windows) Data IIS Web Server ASP.Net SQL Server ArcGIS Technologies HDAS HODM Web Service Web Services Web Service proxies Remote CUAHSI HIS Node (Windows) Data IIS Web Server ASP.Net SQL Server ArcGIS Technologies HDAS HODM Web Service Web Services Web Service proxies Remote CUAHSI HIS Node (Windows) Data IIS Web Server ASP.Net SQL Server ArcGIS Technologies HDAS HODM Web Service Web Services Web Service proxies Remote CUAHSI HIS Nodes (Windows)

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Resource registration Shapefiles TIFF images, GMT rasters Web Services, WMS services Relational databases, ASCII PDFs, URLs “CUAHSI data” NetCDF Coming: Geodatabases and ODM

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D HIS Scalability Adding… …data types and datasets; processing models and services; servers; users and roles – - shall not create unmanageable bottlenecks that require system re- engineering Designing for scalability: Distilling a generic set of web service signatures; resolving semantic and structural heterogeneities Using HODM as a common generic format for time series data, for ease of coding and uniform search interfaces HDAS GUI design to abstract specifics of disparate repositories Leveraging common CI components developed in GEON and other projects Have good design docs, to allow others develop and deploy systems Also: Need to work with agencies to remove the web services bottleneck – some progress with USGS and NOAA!!

SAN DIEGO SUPERCOMPUTER CENTER, UCSD SciR&D Plans HIS 1.0, Oct 30, 06: Version 1 of WaterOneFlow web services; version 1 of ODM; Additional catalogs in HODM; HDAS deployable; HIS workbook; HIS design document Web services and catalogs for fields Further GEON integration: registering ODMs to GEON catalog, and completing a Windows-based node Planning for 5 more years!