Hydrologic Modeling in 2011 David R. Maidment Center for Research in Water Resources University of Texas at Austin Leader of the CUAHSI Hydrologic Information System Project With acknowledgements to Rick Hooper, David Tarboton & Barbara Minsker

Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

Workshop Charge What new technologies for observing, simulating, and tele-communicating will emerge over the next 5-10 years? how will they change the grand challenges for modeling, what will those challenges be? Challenge for this session: –How all the new devices/opportunities emerging in the realm of “cyber-infrastructure”— including, perhaps especially, visualization schemes — might change the way models are developed and applied, including the new kinds of scientific questions to be asked in association with modeling.

Hydrologic Modeling We want to trace the movement of water, chemical and biological constituents through atmospheric, surface and subsurface water We want to do water, mass and energy balances

Hydrologic Information System A system is a connected set of components e.g. University of Texas System A web-based system is a set of components connected using the internet A hydrologic information system (HIS) is a web- based system linking hydrologic databases, tools and models CUAHSI HIS partner institutions

USGS Water Watch System A national hydrologic observing system already exists – CUAHSI adds to it

Real-time Water Quality Estimates Estimated total nitrogen Stream discharge mg/L cfs

CUAHSI Member Institutions 105 Universities as of May 2006

Challenges How to use test-beds to design real WATERS Observatories? How to share data from the test-beds with the whole community? How to include CUAHSI/CLEANER data not collected in the test-beds? How to empower individual scientists? How to make use of petascale computing?

Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

CUAHSI Web Services CUAHSI Web Services Library Web Application: Data Portal Your application Excel, ArcGIS, Matlab Fortran, C/C++, Visual Basic Hydrologic model ……………. Your operating system Windows, Unix, Linux, Mac Internet Simple Object Access Protocol

Rainfall & Snow Water quantity and quality Remote sensing Water Data Modeling Meteorology Soil water

Water Data Web Sites

NWISWeb site output # agency_cd Agency Code # site_no USGS station number # dv_dt date of daily mean streamflow # dv_va daily mean streamflow value, in cubic-feet per-second # dv_cd daily mean streamflow value qualification code # # Sites in this file include: # USGS NEUSE RIVER NEAR CLAYTON, NC # agency_cdsite_nodv_dtdv_vadv_cd USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS USGS Time series of streamflow at a gaging station

CUAHSI Hydrologic Data Access System A common data window for accessing, viewing and downloading hydrologic information USGS NASANCDC EPANWS Observatories

Observation Stations Ameriflux Towers (NASA & DOE)NOAA Automated Surface Observing System USGS National Water Information SystemNOAA Climate Reference Network Map for the US

NWIS Station Observation Metadata Describe what has been measured at this station

Web Page Scraping Programmatically construct a URL string as produced by manual use of the web page Parse the resulting ASCII file

NWIS ArcGIS Excel NCAR Unidata NASA Storet CUAHSI Ameriflux Matlab AccessSAS Fortran Visual Basic C/C++ Some operational services CUAHSI Web Services Data Sources Applications Extract Transform Load

Core Web Methods MethodInputOutput GetSites Obs NetworkAll station codes in network GetSiteInfo Station CodeLat/long, station name GetVariables Obs Network or data source All variable codes GetVariableInfo Variable codeDescription of variable GetValues Station code or lat/long point, variable code, begin date, end date A time series of values GetChart As for GetValueA chart plotting the values

Operational Services Service AmerifluxDaymetMODISNWISNAMHODM Bear Creek GetSites Yes GetSiteInfo Yes GetVariables Yes GetVariableInfo Yes GetValues Yes GetChart Yes

XML Output from GetValues NWIS DayMet MODIS

What is a Data Model A data model is a model that describes in an abstract way how data is represented Data models describe structured data for storage in data management systems such as relational databases. Early phases of many software development projects emphasize the design of a conceptual data model. Lets see what Wikipedia says

CUAHSI Point Hydrologic Observations Data Model A relational database stored in Access, PostgreSQL, SQL/Server, …. Stores observation data made at points Consistent format for storage of observations from many different sources and of many different types. Streamflow Flux tower data Precipitation & Climate Groundwater levels Water Quality Soil moisture data

Hydrologic Observations Data Model (HODM)

Serving investigator data Several choices –You build CUAHSI compatible services from your database –You copy data into the HODM and use CUAHSI services –You copy your data to an HODM and it is served from SDSC Your database Your implementation of CUAHSI services HODM Standard CUAHSI services

Modeling Services Simulation models can be packaged as web services They can be queried and provide responses just like data archives We have an integrated network of data sources and models A big challenge to integrate all the data streams!

Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them Objective NWIS NARR NAWQA NAM-12 request request return return What we don’t want ….. Michael Piasecki Drexel University

Semantic Mediator What we do want ….. NWIS NAWQA NARR generic request request HODM Michael Piasecki Drexel University

Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

Regional Storm Water Modeling Program and Master Plan for San Antonio City of San Antonio

Modeling System Rainfall Data: Rain gages Nexrad Calibration Data: Flows Water Quality Geospatial Data: City, County SARA, other Floodplain Management Integrated Regional Water Resources planning Capital Improvement Planning Flood Forecasting Water quality planning San Antonio Regional Watershed Modeling System “Bring the models together”

Database Geo- HMS Geo- RAS GIS- Gflow Interface HEC-HMS HEC-RAS Gflow GIS GIS Preprocessors for Hydrologic Models

Interface data models HMS RAS Gflow GIS Geo Database Arc Hydro data model Connecting Arc Hydro and Hydrologic Models

Digital Rain Maps from National Weather Service (03/04/2004)

FEMA 100-year flood plain map in Bexar County

Regional Watershed Modeling System Case Study Rosillo Creek watershed Arc Hydro Geodatabase for whole watershed HEC-HMS hydrology model for whole watershed HEC-RAS hydraulic model for Rosillo Creek Salado Creek watershed Components: Bexar County

Arc Hydro and HEC-HMS Arc Hydro Schematic Network HEC-HMS Hydrologic Model Calculates Flows

Arc Hydro and HEC-RAS Arc Hydro Channel Cross Sections HEC-RAS Hydraulic Model Calculates Water Surface Elevations

Flow Change Points Models communicate with one another through Arc Hydro at designated points

Nexrad Map to Flood Map in Arc 9 Model Builder FLO ODP LAIN MAP Flood map as output Model for flood flow Model for flood depth HMS Nexrad rainfall map as input

Web-Accessible Regional Watershed Modeling System Complete storage of simulation models and workflows in geodatabases

Hydrologic Modeling in 2011 The charge and challenges Hydrologic information system – web services Integrating models and data using scientific workflows Hydrologic Observing System

CUAHSI Hydrologic Observing System Continental US Scale (coast to coast data coverage, HIS-USA) 1:500,000 scale Regional Scale (e.g. Neuse basin) 1:100,000 scale Watershed Scale (e.g. Eno watershed ) 1:24,000 scale Site Scale (experimental site level) Site scale Multiscale information delivery A multiscale web portal system for observing and interpreting hydrologic phenomena by integrating data and models for any location or region in the United States PointPoint Observation Scale (gage, sampling location) North American Scale (e.g. North American Regional Reanalysis of climate) 1:1,000,000 scale

GeoTemporal Reference Frame A defined geospatial coordinate system for (x,y,z) A defined time coordinate system (UTC, Eastern Standard Time, ….) A set of variables, V Data values v(x,y,z,t) Space (x,y,z) Time, t Variables, V v – data values Data Cube

Series and Fields Features Point, line, area, volume Discrete space representation Series – ordered sequence of numbers Time series – indexed by time Frequency series – indexed by frequency Surfaces Fields – multidimensional arrays Scalar fields – single value at each location Vector fields – magnitude and direction Random fields – probability distribution Continuous space representation

mm / 3 hours Precipitation Evaporation North American Regional Reanalysis of Climate Variation during the day, July 2003 NetCDF format

Continuous Space-Time Model – NetCDF (Unidata) Space, L Time, T Variables, V D Coordinate dimensions {X} Variable dimensions {Y}

Space, FeatureID Time, TSDateTime Variables, TSTypeID TSValue Discrete Space-Time Data Model ArcHydro

Hydrologic Flux Coupler Precipitation Evaporation Streamflow Define the fluxes and flows associated with each hydrovolume Groundwater recharge See Chapter 9 of Status Report for Details

ArcGIS ModelBuilder Application for Automated Water Balancing Fields Series Geospatial

Water Resource Regions and HUC’s

NHDPlus for Region 17E

NHDPlus Reach Catchments ~ 3km 2

Reach Attributes Slope Elevation Mean annual flow –Corresponding velocity Drainage area % of upstream drainage area in different land uses Stream order

Ingestion of real-time streamflow data A national hydrologic observing system already exists – CUAHSI adds to it

Continental Water Dynamics Model Hydrologic Information System Hydrologic Observing System Hydrologic Modeling System

Petascale Computing 2.6 million river reaches on a 1:100,000 scale map of continental US Solve continuity and momentum equations once on each reach (~ 5.2 million equations) takes ~ 200 parallel processors Pittsburgh Supercomputer Center has 3000 parallel processors It is within reach to simulate flows on all reaches continuously through time with data assimilation from gaging stations

Conclusions Web services support a web-based hydrologic information system connnecting data, tools and models Models can be configured as web services Scientific workflows automate the integration of components A continental water dynamics model is feasible