CUAHSI Hydrologic Information Systems David R. Maidment and Ernest To Center for Research in Water Resources, University of Texas at Austin Hydrosystems Laboratory University of Illinois at Urbana-Champaign, 18 August 2006

CUAHSI Hydrologic Information Systems Introduction HIS Server CUAHSI web services Demo by of Corpus Christi Bay by Ernest To Data models and some longer range thinking

CUAHSI Hydrologic Information Systems Introduction HIS Server CUAHSI web services Demo of Corpus Christi Bay by Ernest To Data models and some longer range thinking

Ocean Sciences CUAHSI-Hydrologic Information Systems CUAHSI – Consortium of Universities for the Advancement of Hydrologic Science, Inc Formed in 2001 as a legal entity Program office in Washington (5 staff) Supported by the National Science Foundation Earth Sciences Atmospheric Sciences UCAR CUAHSI Unidata HIS National Science Foundation Geosciences Directorate

CUAHSI Member Institutions 115 Universities as of August 2006

Common Vision: WATERS Network Sensors and Measurement Facility Synthesis Informatics Observatories/ Environmental Field Facilities A combined CLEANER-CUAHSI effort

CUAHSI Hydrologic Information Systems Introduction HIS Server CUAHSI web services Demo of Corpus Christi Bay by Ernest To Data models and some longer range thinking

Digital Watershed How can hydrologists integrate observed and modeled data from various sources into a single description of the environment?

Digital Watershed Hydrologic Observation Data Geospatial Data Weather and Climate Data Remote Sensing Data (NetCDF) (GIS) (Relational database) (EOS-HDF) Digital Watershed A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather and climate data into a connected database for a hydrologic region

HIS Servers Hydrologic Observations Server GIS Data Server Weather and Climate Server Remote Sensing Server Digital Watershed HIS Servers provide hydrologic observations, weather and climate, GIS and remote sensing data. For HIS version 1.0, the focus is a hydrologic observations server for data from gages and monitoring sites at point locations.

CUAHSI Hydrologic Information System Levels National HIS – San Diego Supercomputer Center Workgroup HIS – research center or academic department Personal HIS – an individual hydrologic scientist HIS Server HIS Analyst Map interface, observations catalogs and web services for national data sources Map interface, observations catalogs and web services for regional data sources; observations databases and web services for individual investigator data Application templates and HydroObjects for direct ingestion of data into analysis environments: Excel, ArcGIS, Matlab, programming languages; MyDB for storage of analysis data

HIS Server Supports data discovery, delivery and publication –Data discovery – how do I find the data I want? Map interface and observations catalogs –Data delivery – how do I acquire the data I want? Use web services or retrieve from local database –Data Publication – how do I publish my observation data? Use Observations Data Model

Observations Catalog Specifies what variables are measured at each site, over what time interval, and how many observations of each variable are available

HIS Server Architecture Map front end – ArcGIS Server 9.2 (being programmed by ESRI Water Resources for CUAHSI) Relational database – SQL/Server 2005 or Express Web services library – VB.Net programs accessed as a Web Service Description Language (WSDL)

National and Workgroup HIS National HIS has a polygon in it marking the region of coverage of a workgroup HIS server Workgroup HIS has local observations catalogs for coverage of national data sources in its region. These local catalogs are partitioned from the national observations catalogs. For HIS 1.0 the National and Workgroup HIS servers will not be dynamically connected. National HISWorkgroup HIS

Point Observations Information Model Data Source Network Sites Variables Values {Value, Time, Qualifier} USGS Streamflow gages Neuse River near Clayton, NC Discharge, stage (Daily or instantaneous) 206 cfs, 13 August 2006 A data source operates an observation network A network is a set of observation sites A site is a point location where one or more variables are measured A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time A qualifier is a symbol that provides additional information about the value

Data Discovery and Delivery Data Source Network Sites Variables Values Observations metadata Observations data HIS Server Observations Catalog Web services Data Discovery Data Delivery HIS facilitates data discovery by building and maintaining observations catalogs Data delivery occurs through web services from remote data archives or local observations databases. Water resource agencies support data delivery services.

CUAHSI Hydrologic Information Systems Introduction HIS Server CUAHSI web services Demo of Corpus Christi Bay by Ernest To Data models and some longer range thinking

Web Services with HIS Server Publication services for local observations databases Ingestion Services for remote data archives Send data out from the server Enable users to access data in remote archives

Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

Water Data Web Sites

NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS Time series of streamflow at a gaging station USGS has committed to supporting CUAHSI’s GetValues function

Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

Water Quality Measurement Sites in EPA Storet Substantial variation in data availability from states Data from Bora Beran, Drexel University

Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

Geographic Integration of Storet and TCEQ Data in HIS

CUAHSI Hydrologic Data Access System A common data window for accessing, viewing and downloading hydrologic information USGS NASANCDC EPANWS Observatory Data Arc Hydro Server will be a customization of ArcGIS Server 9.2 for serving water observational data

NWIS ArcGIS Excel NCAR Unidata NASA Storet NCDC Ameriflux Matlab AccessJava Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load

CUAHSI Web Services Web Services Library Web Application: Data Portal Your application Excel, ArcGIS, Matlab Fortran, C/C++, Visual Basic Hydrologic model ……………. Your operating system Windows, Unix, Linux, Mac Internet Simple Object Access Protocol

CUAHSI Hydrologic Information Systems Introduction HIS Server CUAHSI web services Demo of Corpus Christi Bay by Ernest To Data models and some longer range thinking

Example: Corpus Christi Bay Environmental Info System Workgroup HIS implementation Uses ODM to store hydrology and environmental data from state agencies and academic investigators. Contains web-services to regional data repositories (e.g. TCOON). Water quality data sites in Corpus Christi Bay (maps by Tyler Jantzen) Demo: TXHIS ODM webservice

ODM (Observations Data Model) = Observations Catalog + Values Table +Metadata Tables

ExcelCUAHSI Web service How Excel connects to ODM Obtains inputs for CUAHSI web methods from relevant cells. Available Web methods are GetSiteInfo, GetVariableInfo GetValues methods. converts standardized request to SQLquery. imports VB object into Excel and graphs it converts response to a standardized XML. Observations Data Model SQL query Response HydroObjects converts XML to VB object parses user inputs into a standardized CUAHSI web method request.

CUAHSI Hydrologic Information Systems Introduction HIS Server CUAHSI web services Demo of Corpus Christi Bay by Ernest To Data models and some longer range thinking

Series and Fields Features Point, line, area, volume Discrete space representation Series – ordered sequence of numbers Time series – indexed by time Frequency series – indexed by frequency Surfaces Fields – multidimensional arrays Scalar fields – single value at each location Vector fields – magnitude and direction Random fields – probability distribution Continuous space representation

mm / 3 hours Precipitation Evaporation North American Regional Reanalysis of Climate Variation during the day, July 2003 NetCDF format

Space, L Time, T Variable, V D Data Cube – What, Where, When “What” “Where” “When” A data value

Continuous Space-Time Data Model -- NetCDF Space, L Time, T Variables, V D Coordinate dimensions {X} Variable dimensions {Y}

Space, FeatureID Time, TSDateTime Variables, TSTypeID TSValue Discrete Space-Time Data Model

Geostatistics Time Series Analysis Multivariate analysis Hydrologic Statistics How do we understand space-time correlation fields of many variables?

Water OneFlow Like Geospatial OneStop, we need a “Water OneFlow” – a common window for water data and models Advancement of water science is critically dependent on integration of water information Federal Academic Local State

Conclusions This is a complex and important problem that will not be solved soon Web services architecture will work and is valuable Major water agencies are buying into our web services design, in particular the USGS We need to think more deeply and abstractly about the way data is used to represent water and the water environment