1. CUAHSI Observations Data Model A relational database stored in Access, PostgreSQL, SQLServer, …. Stores observation data made at points Access data through web interfaces Fill using automated data harvesting Streamflow Flux tower data Precipitation & Climate Groundwater levels Water Quality Soil moisture data

2. Purposes Hydrologic Observations Data System to Enhance –Retrieval –Integrated Analysis –Multiple Investigators Standard and Scalable Format for Sharing Ancillary information (metadata) to allow unambiguous interpretation and use – incorporating uncertainty Traceable heritage from raw measurements to usable information – quality control levels Premise A relational database at the single observation level (atomic model) –Querying capability –Cross dimension retrieval and analysis

3. Community Design Requirements (from comments of 22 reviewers) Incorporate sufficient metadata to identify provenance and give exact definition of data for unambiguous interpretation Spatial location of measurements Scale of measurements (support, spacing, extent) Depth/Offset Information Censored data Classification of data type to guide appropriate interpretation –Continuous –Indication of gaps Indicate data quality

4. Scale issues in the interpretation of data The scale triplet From: Blöschl, G., (1996), Scale and Scaling in Hydrology, Habilitationsschrift, Weiner Mitteilungen Wasser Abwasser Gewasser, Wien, 346 p. a) Extentb) Spacing c) Support

5. From: Blöschl, G., (1996), Scale and Scaling in Hydrology, Habilitationsschrift, Weiner Mitteilungen Wasser Abwasser Gewasser, Wien, 346 p.

6. Ernest To Center for Research in Water Resources University of Texas at Austin 20061011 What are the basic attributes to be associated with each single observation and how can these best be organized? A data source operates an observation network A network is a set of observation sites Data Source and Network SitesVariablesValuesMetadata Depth of snow pack Streamflow Landuse, Vegetation Windspeed, Precipitation Data Delivery Controlled Vocabulary Tables e.g. mg/kg, cfs e.g. depth e.g. Non-detect,Estimated, A site is a point location where one or more variables are measured Metadata provide information about the context of the observation. A variable is a property describing the flow or quality of water A value is an observation of a variable at a particular time Data Discovery Hydrologic Observations Data Model See http://www.cuahsi.org/his/documentation.htmlhttp://www.cuahsi.org/his/documentation.html

7. Hydrologic Observations Data Model What are the basic attributes to be associated with each single observation and how can these best be organized? See CUAHSI Community Observations Data Model Working Design Specifications Document http://www.cuahsi.org/his/documentation.html

8. 1 1 CouplingTable SiteID (GUID) HydroID (Integer) MonitoringPoint SiteID SiteCode SiteName Latitude Longitude … Hydrologic Observations Data Model 1 1 OR Independent of, but coupled to Geographic Representation HODM Arc Hydro

9. Variable attributes VariableName, e.g. discharge VariableCode, e.g. 0060 SampleMedium, e.g. water Valuetype, e.g. field observation, laboratory sample IsRegular, e.g. Yes for regular or No for intermittent TimeSupport (averaging interval for observation) DataType, e.g. Continuous, Instantaneous, Categorical GeneralCategory, e.g. Climate, Water Quality NoDataValue, e.g. -9999 m 3 /s L 3 /T Cubic meters per second

10. Data Types Continuous (Frequent sampling - fine spacing) Instantaneous (Spot sampling - coarse spacing) Cumulative Incremental Average Maximum Minimum Constant over Interval Categorical

11. Groups and Derived From Associations

12. Stage and Streamflow Example

13. Daily Average Discharge Example Daily Average Discharge Derived from 15 Minute Discharge Data

14. Offset OffsetValue Distance from a datum or control point at which an observation was made OffsetType defines the type of offset, e.g. distance below water level, distance above ground surface, or distance from bank of river

15. Water Chemistry from a profile in a lake

16. Methods and Samples Method specifies the method whereby an observation is measured, e.g. Streamflow using a V notch weir, TDS using a Hydrolab, sample collected in auto-sampler SampleID is used for observations based on the laboratory analysis of a physical sample and identifies the sample from which the observation was derived. This keys to a unique LabSampleID (e.g. bottle number) and name and description of the analytical method used by a processing lab.

17. Accuracy and Precision ObsAccuracyStdDev Numeric value that expresses measurement accuracy as the standard deviation of the error associated with each specific observation

18. Observation Series An Series is a set of all the observations of a particular variable at one place, i.e. with unique SiteID. The ObservationSeriesCatalog is programmatically generated to provide a means by which a user can get simple descriptive information about the variables observed at a location.

19. Data Quality Qualifier Code and Description provides qualifying information about the observations, e.g. Estimated, Provisional, Derived, Holding time for analysis exceeded QualityControlLevel records the level of quality control that the data has been subjected to. - Level 0. Raw Data - Level 1. Quality Controlled Data - Level 2. Derived Products - Level 3. Interpreted Products - Level 4. Knowledge Products

20. 15 min Precipitation from NCDC

21. Irregularly sampled groundwater level

22. ODM and HIS in an Observatory Setting e.g. http://www.bearriverinfo.org

23. ODM and HIS in an Observatory Setting Integration of Sensor Data With HIS Observations Database (ODM) Base Station Computer(s) Data Processing Applications Internet Telemetry Network Sensors Data discovery, visualization, analysis, and modeling through Internet enabled applications Programmer interaction through web services Internet Workgroup HIS Tools Web Server

24. Available National and Workgroup HIS Tools Observations Database (ODM) Internet Data discovery, visualization, analysis, and modeling through Internet enabled applications Programmer interaction through web services Web Server Other National Databases NWIS, STORET, Etc.

25. Observations Database (ODM) User Interaction through Web Browser SQL Queries passed from Time Series Viewer to the server database Query results can be exported to a browser window or directly to Microsoft Excel Time Series Analyst http://water.usu.edu/analyst/

26. Automated Ingestion of Sensor Data into ODM Observations Database (ODM) Base Station Computer(s) Data Processing Applications Telemetry Network Sensors Internet Challenges Heterogeneity Establishing standards Sensor/system descriptions Sensor ML

27. Data Distribution Via XML Web Services Machine to machine communication of data over the internet Users can program against database as if it were on their local machine Replace SQL queries to database with calls to the appropriate web service

28. WaterOneFlow Web Services Data accessthrough web services Data storage through web services Downloads Uploads Observatory data servers CUAHSI HIS data servers 3 rd party data servers e.g. USGS, NCDC GIS Matlab IDL Splus, R Excel Programming (Fortran, C, VB) Web services interface HDAS Website Portal and Map Viewer Information input, display, query and output services Preliminary data exploration and discovery. See what is available and perform exploratory analyses HTML -XML WSDL - SOAP

29. Example: Matlab use of CUAHSI Web Services % create ODM Web Service Class createClassFromWsdl('http://water.usu.edu/BRODM/brodm.asmx?WSDL'); % Discover the sites available xmlSites=GetSites(instODM); strSites=xml_parse(xmlSites); Nsites=length(strSites.Sites) … % Get information about a selected site xmlSiteInfo=GetSiteInfo(instODM,scode); strSiteInfo=xml_parse(xmlSiteInfo); … % Get the data values xmlValues = GetValues(instODM,scode,varcode,D1i,D2i); strValues=xml_parse(xmlValues); … % Plot the data plot(dt,vals); datetick % Axis label from metadata from web service return ylabel(strSiteInfo.Variables(var).Variable.VariableName)

30. ODM Next Steps Beta test in test beds (unanticipated requirements) Unit conversions Additional data types (vector, grid) Tools, incl. Data Loader Integration with search [Moving sensors (FerryMon)]

31. Managing Data Within ODM - ODM Tools Load – import existing data directly to ODM Query and export – export data series and metadata Visualize – plot and summarize data series Edit – delete, modify, adjust, interpolate, average, etc.