1. WQX Data Flow Expansion Project
Ute Mountain Ute
WQX Data Flow Expansion Project
Region 7 / Region 8 Exchange Network Meeting
Lenexa, KS EPA Region 7 Office
September 26th and 27th

2. Project Overview From the EPA 2014 Grant Solicitation:
“Provide participants improved exchange of and access to high quality water quality data.”
The Ute Mountain Ute Tribe (UMUT) along with 19 other Tribal Environmental Programs in EPA Region VIII have been successfully utilizing the AWQMS application to manage and flow their water quality data. This effort has been a success not only for the region but for Tribal programs across the country that have benefited from the capabilities of AWQMS developed for these programs. This project is designed to build on past success by providing training and support to build Tribal capacity to increase regional submission of WQX data via the Exchange Network and to publish data with the virtual node technology while also providing enhancements to increase the quantity and quality of data flow with the ability to flow habitat, macroinvertebrate and continuous monitoring data.

3. Project Overview
In Fall, 2013 Ute Mountain Ute Tribal Water Quality Program applied for an FY-2014 Cycle Exchange Network Grant.
Contracted with Gold Systems to deliver many of the “outputs” associated with the grant goals.
Scott Clow - Environmental Programs Director
Colin Larrick – Grant/Contract Manager
Although the Region VIII AWQMS implementation has been a significant success, there are still many enhancements needed to allow all of the tribes in the region to flow not only their water chemistry data but also their biological and habitat assessment data. Additionally, many tribal programs in the region have begun to collect and store continuous monitoring data. AWQMS needs to be enhanced to better manage this data locally and submit this data to WQX. This project will address these deficiencies by creating and providing enhancements to AWQMS to support habitat and benthic macroinvertebrate data as well as for continuous monitoring data while at the same time provide technical support and coordination between Tribes in the region to build capacity and significantly increase regional submissions of WQX data via the exchange network.

4. Project Overview
Directly benefits all tribal water quality programs in EPA Region 8
Indirectly benefits approximately 60+ other tribes using AWQMS
Indirectly benefits 7 states using AWQMS
Indirectly benefits approximately 65+ volunteer and/or municipal groups in the Colorado Data Sharing Network
As with previous efforts, the enhancements made to the AWQMS system will benefit not only the Region VIII tribes but also the many other AWQMS users across the nation.

5. 23 of the Federally Recognized Tribes in EPA Region 8 are participants
Source for graphic:

6. Project Overview
The Ambient Water Quality Monitoring System (AWQMS) is a key component of the project.
The Ambient Water Quality Monitoring System (AWQMS) is a key component of the project. AWQMS provides a single tool for consolidation, validation, analysis, assessment, and sharing of the data.

7. Project Overview
Goal 1: Significantly increase regional submissions of WQX data via the Exchange Network
Over the past several years, almost all of the Region VIII tribes have been managing their data in a WQX compatible format and sharing this data with Regional EPA personnel. Only about a third of these tribes, however, have submitted this data to WQX for inclusion in the STORET Data Warehouse and other national data publishing tools. This goal is to increase tribal submissions in the region to 100% of participating tribes by improving the tribe's ability to filter the data in AWQMS that is submitted to WQX. By adding capabilities for tribes to select specific characteristics to submit (such as their required 106 parameters) and to select certain sensitive locations to exclude, we will enable the remaining regional tribes to submit their required data while protecting otherwise sensitive information. Flowing water quality data to EPA via the WQX Exchange Network flow is a tier two priority.

8. Project Overview
1.1 Update the AWQMS application to provide additional filters on data included in WQX submissions and allow these filter criteria to be stored so that it's simpler to submit to WQX
 Complete!
1.1 Update the AWQMS application to provide additional filters on data included in WQX submissions and allow these filter criteria to be stored so that it's simpler to submit to WQX.

9. Project Accomplishments

10. Project Accomplishments

11. Project Accomplishments

12. Project Accomplishments

13. Project Accomplishments

14. Project Overview
1.2 Provide a webinar to the region VIII tribes on the enhanced WQX submission features in AWQMS. Stress the importance of submitting required 106 parameters.
 Complete!
1.2 Provide a webinar to the region VIII tribes on the enhanced WQX submission features in AWQMS. Stress the importance of submitting required 106 parameters.

15. Project Accomplishments

16. Project Overview
Goal 2: Leverage the EPA-hosted Virtual Node for data publishing services in AWQMS
Our existing AWQMS system will be enhanced to take advantage of the new Virtual Node technology to allow our consortium members to publish their data. Utilization of the Virtual node for data publishing is a tier one priority.

17. Project Overview 2.1 Review & Plan for VN Publishing 
2.2 Develop Services 
2.3 Provide a Webinar on VN Services 
 Complete!
2.1 Review available documentation on the virtual node, collaborate with virtual node personnel and develop a plan for implementing data publishing services in AWQMS.
2.2 Develop data publishing services that implement the virtual node.
2.3 Provide a webinar to the Region VIII Tribes on the benefits and use of the new virtual node services.

18. Project Accomplishments
Added AWQMS Virtual Node Publishing Services
getProjects
getLocations
getResults
getMetrics
getIndexes

19. Project Accomplishments

20. Project Overview
Goal 3: Flow Habitat Assessment and Benthic Macroinvertebrate Data to WQX
The region VIII tribes are expanding their water quality programs to capture and analyze these additional forms of water quality data and must expand their use of AWQMS in order to manage, analyze, and submit high quality data to WQX.
This goal expands on the successful output of a previous grant that added capabilities to AWQMS for managing these data sets and supported initial tribal programs in loading this data into AWQMS and flowing it to WQX. Under this grant, this effort will be expanded to include participation by many additional tribes in the region.
As part of this effort to increase the use of these datasets, the AWQMS data analysis (reporting, graphing, and mapping) will be enhanced to support these new data sources. Additionally AWQMS will be enhanced to automatically generate certain basic metrics based on existing macroinvertebrate or other population census result data in the system.  The changes would allow each organization using AWQMS to pre-populate their own profile per taxonomic name and then run routines that would generate and store certain metrics based on the existing result data in accordance with the profile for each taxon.  (e.g. calculate a “Percent Clingers” metric).  These enhancements to AWQMS will allow the tribal program to utilize its water quality data to address environmental and human health concerns on and near their tribal lands and will assist the tribes to meet the reporting and data submission requirement outlined by the EPA.
Flowing these expanded data sets and calculated metrics to EPA via WQX is a tier two priority.

21. Project Overview 3.1 HA and Macro Data Inventory 
3.2 Provide Webinars – Ongoing
(Spring 2018)
3.3 Provide Assistance – Ongoing
3.4 Analysis Tools Enhancements– Started
3.5 Basic Abundance Metrics - Analyzing
3.6 Share Enhancements - Started
3.1 Perform an inventory to assess and catalog which of the Region VIII Tribes are collecting Habitat Assessment and/or Macroinvertebrate data and what formats their data are in.
3.2 Provide one or more webinars to the Region VIII Tribes on importing or entering habitat assessment and/or macroinvertebrate data into AWQMS.
3.3 Assist tribes with mapping their data to the WQX data elements and submitting their initial macroinvertebrate and habitat assessment data to WQX.
3.4 Enhance AWQMS to provide needed analysis tools (statistical reports and/or graphs) specific to macroinvertebrate and habitat assessment data.
3.5 Generate metrics based on population census data.
3.6 Share AWQMS enhancements with current AWQMS community members.

22. Project Accomplishments

23. Project Accomplishments

25. Project Accomplishments

26. Project Accomplishments

27. Project Accomplishments

28. Project Accomplishments

29. Project Overview
Goal 4: Support Continuous Water Quality Monitoring Data and Flow Summary Data to WQX
Currently, the AWQMS database is based on the WQX data model and is, therefore, not currently optimized for handling massive amounts of continuous data. The tribes desire to enhance the existing AWQMS database to more efficiently store, retrieve, and share continuous water quality monitoring data. Additionally, the tribes need improved tools for analyzing their continuous data. Lastly, the tribes will use improved features of AWQMS to submit summarizations of continuous data (e.g. daily average, min, and max) to WQX. Flowing these expanded data sets to EPA via WQX is a tier two priority.

30. Project Overview 4.1 Optimize database and data access 
4.2 Optimize data analysis tools 
4.3 Improve data summary submissions 
4.4 Share the improvements 
 Complete!
4.1 Optimize the Region VIII AWQMS system to better support continuous data.
4.2 Provide data analysis tools (graphs, reports, and mapping tools) in AWQMS optimized for continuous monitoring data.
4.3 Utilize AWQMS to submit summary data to WQX (as recommended by EPA).
4.4 Share AWQMS continuous monitoring data improvements with current AWQMS community members.

31. Project Accomplishments

32. Project Accomplishments

33. Project Accomplishments

34. Project Accomplishments

35. Project Accomplishments

36. Project Accomplishments

37. Goal 5: Building Tribal Capacity
Project Overview
Goal 5: Building Tribal Capacity
In an effort to efficiently train tribal members on the use of the new and existing AWQMS capabilities, a combination of workshops and re-usable online training modules will be utilized. Under this goal, two regional workshops will be provided to give tribal personnel hands on training using these new capabilities. These workshops would be held at Ute Mountain and EPA Region VIII offices.
In addition to the workshops, several online training modules will be developed allowing the tribes to learn and/or review core aspects of the AWQMS application. These will be available on demand for the tribal programs and will be critical for tribes that have high turnover in the water quality data management positions.
Finally, this goal will also provide direct support from the AWQMS contractors for issues not addressed in the workshops or online training.

38. Project Overview 5.1 Provide 2 Workshops – Planning (Spring 2018)
5.2 Online Training Tools - Started
5.3 Direct Support - Ongoing
5.1 Prepare for and provide two workshops to give tribal personnel hands on experience working with the new capabilities added under this grant.
5.2 Develop online training courses to allow tribal personnel to obtain on demand instruction on core aspects of water quality data management and the use of AWQMS.
5.3 Provide direct contractor support for tribes for issues not covered in the workshops or online training.

39. Project Accomplishments
Assist tribes
Import Data into AWQMS
Flow data to WQX via the Exchange Network
Configure and use AWQMS QC Thresholds

40. Project Accomplishments
Updated User Guide (In Review)
Training Video for Auto-Submit Feature

41. Project Accomplishments

42. Project Accomplishments
Before: only 1/3 of participating tribes were submitting data to WQX
Now all participating tribes have submitted something to WQX

43. Project Accomplishments
Total Activities: 64,639
Total Results: 627,073
Results Last 90 Days: 25,262
Results YTD: 49,978
Results Last 12 Months: 134,092
16 Tribes have submitted data in 2017

44. Project Next Steps Continue Training & Webinars
Continue Remote Assistance
Release Updated User Guide
Additional Data Analysis Improvements

45. Project Next Steps Taxa Profile Capability Generate Abundance Metrics
Update Training Videos
R8 Tribal Webinar (or workshop?) in Spring 2018

46. Other Recent AWQMS Work
Leverage EPA Water Quality Portal for combined data analysis
GIS “Web Feature Service” (WFS) for integration with GIS tools

47. GIS Centric Web Services
ArcGIS
QGIS
Mark – Here are some recent examples of using Open Geospatial Consortium-compliant tools to obtain information from AWQMS using the new GIS web feature service. On the right is QGIS, an Open Source OGC compliant GIS tool. On the left is the ESRI ArcGIS. Both obtained the information from the same AWQMS Web Feature Service following OGC standards.
Really quick – Almost click through this.

48. Other Recent AWQMS Work
Added QA tools for identifying & correcting inconsistent metadata “en masse”
Characteristic Name
Activity Type
Activity Collection Method
Activity Time Zone (Start & End)

49. Other Recent AWQMS Work
Added QA tools for identifying & correcting inconsistent metadata “en masse”
Activity Equipment & Comment
Activity Collection Method
Activity Media and Subdivision
Result Value Type

50. Batch Update Capability

51. Batch Update Capability

52. Batch Update Capability

53. Batch Update Capability

54. Batch Update Capability

55. Batch Update Capability

56. Other Recent AWQMS Work
Auto-Sync AWQMS Domain Values Lists with WQX Domain Value Lists
Includes All WQX domain lists - Nightly
We rarely manually update AWQMS DVLs anymore
Drastically increased our WQX submission success rate!!
Mark
One of the really nice things about AWQMS is that it uses the same Domain Value Lists as WQX does to govern data entry for data elements that need to be entered in a consistent manner. That really simplifies the process and reduces submission errors when sending data to WQX. Another really nice feature that this EN project has implemented is a nightly auto-synchronization of domain value lists from WQX to AWQMS using the WQX Domain Value List web services. This has drastically increased WQX submission successes from not only the three states involved in the project, but also hundreds of other programs and groups using AWQMS.

57. Other Upcoming AWQMS Work
Auto-Calc Metals Standards for Data Analysis (Starting Soon)
E.g., based on Hardness
Previous input from the state of UT DWQ and EPA Region 8
More input needed.
Mark
Earlier, I briefly alluded to the ability in AWQMS to compare actual data against water quality standards in order to perform exceedance analysis. AWQMS currently allows you to record and maintain static chronic, acute, and other water quality standards per parameter and designated use combination. Project goal number four is to implement a feature to auto-calculate dynamic water quality standards for metals and possibly other parameters needing dynamically calculated standards based on other parameter results returned from a sampling activity. We’ve previously received some good input on this from the Utah Division of Water Quality and from EPA Region 8 tribal water quality support leads. We’ll be consolidating those notes soon and then reviewing them with the states involved in the project. We’ll also solicit input from other AWQMS community members as well.

58. Other Upcoming AWQMS Work
Data analysis improvements / better support
Biological Data
Habitat Assessment Data
Continuous Data
Improved SP-14b support (for tribes)

59. Other Upcoming AWQMS Work
Document “Library”
Single place to search for documents attached to Locations, Projects, Activities, Results, Organization
Different types of searches
Different ways to categorize
Potentially also useful for consortium-wide resources

60. Other Upcoming AWQMS Work
Improved Habitat Assessment Data Support
Based on NRSA forms (and database)
Based on experience with tribes’ forms and spreadsheets
Some input from OK WRB already
Possible input coming from OR
More input needed

61. Other Upcoming AWQMS Work
Assessments
Improved meta-data management
Improved Assessment Decision Support

62. Other Upcoming AWQMS Work
Assessments
Local Assessment Management System – Tied to WQ Data
Capture Historical Basis
Submit Biennial Report to ATTAINS via Exchange Network

63. Other Upcoming AWQMS Work
Continuous Monitoring Data
Dataset “adjustments” (e.g. drift)
EPA Central Sensor Catalog
Become a “Data Appliance”

64. Other Upcoming AWQMS Work
Integrated Data Review Process
Review dataset
Software assisted identification of potential issues
Individual or batch adjudication

65. AWQMS Strategic Plan Event Analysis Capture and manage “events”
During data analysis be able to visualize data in context of events

66. AWQMS Strategic Plan Open Dumps Tracking (Tribes)
Download data via Exchange Network
Visualize on Map in context of monitoring locations and/or beaches

67. Thank you for listening!
Scott Clow
Environmental Programs Director
Ute Mountain Ute Tribe
 (mobile)
Colin Larrick
Water Quality Specialist
Ute Mountain Ute Tribe
Environmental Programs Department
(970)
(970)  (cell)
Mark LeBaron
Director, Project Management
Sr. Systems Analyst
Gold Systems, Inc.

68. Supplemental Slides

69. AWQMS: An Exchange Network Success Story
More Result data in WQX via AWQMS than via any other single source
17% of Result data in WQX flowed to WQX via AWQMS
The following statistics are as of January, We are confident they’d be even more impressive with today’s updated numbers
More Result data in WQX via AWQMS than via any other single source
17% of Result data in WQX flowed to WQX via AWQMS

70. AWQMS: An Exchange Network Success Story
73 tribal organizations flow data to WQX using AWQMS.
31% of tribal organizations that flow data to WQX, are using AWQMS
7 states currently flow data to WQX using AWQMS
73 tribal organizations flow data to WQX using AWQMS.
31% of tribal organizations that flow data to WQX, are using AWQMS
7 states currently flow data to WQX using AWQMS

71. AWQMS: An Exchange Network Success Story
188 organizations use AWQMS to flow data to WQX.
26% of Organizations that flow data to WQX, are using AWQMS
Newer numbers will be more impressive!
188 organizations use AWQMS to flow data to WQX.
26% of Organizations that flow data to WQX, are using AWQMS

72. The AWQMS Big Picture Real World Grab Samples Habitat Assessments
Probe Readings
Fish Census Counts
Benthic M.I. Census Counts
Etc., etc.
Some people may wonder how AWQMS fits into the big “WQX” and “Water Quality Portal” picture.
In the real world, we collect our various types of samples and observations; and we record results or receive results back from our labs.

73. The AWQMS Big Picture Real World Grab S. Habitat As. Probe S.
Fish Census
Benthic M.I.
Census
Etc.
(click through this to the next slide)

74. The AWQMS Big Picture “Data Limbo” Laptop? Lab  .PDF Network Server?
Real World
Grab S.
Habitat As.
Probe S.
Fish Census
Benthic M.I.
Census
Etc.
“Data Limbo”
Laptop?
Lab  .PDF
Network Server?
Lab  Excel
Lab  CSV
Paper sheets
Probe  CSV
Flash Drive?
Depending on your situation you might next experience a sort of “data limbo” where there seems to be all sorts of data files, file types and formats, even paper field sheets and notebooks. Depending on your processes and mechanisms, it might seem like the data can be stored all over the place. (example of one group who contacted Gold Systems because they had put a thumb drive with all their data through the wash).

75. The AWQMS Big Picture Real World Grab S. Habitat As. Probe S.
Fish Census
Benthic M.I.
Census
Etc.
“Data Limbo”
Lab  .PDF
Lab  Excel
Lab  CSV
Flashdrive
Paper sheets
Probe  CSV
Flash Drive?
Laptop?
Network Srvr?
(Click through this slide)

76. The AWQMS Big Picture Local Information AWQMS All Your Data
Local Analysis Tools
Local Data Verification
Local Data QA/QC
Local Decision Support
Local Data Assessment
Control what is sent to WQX and when it is sent
Local Portal (Optional)
The AWQMS Big Picture
Real World
Grab S.
Habitat As.
Probe S.
Fish Census
Benthic M.I.
Census
“Data Limbo”
Lab  .PDF
Lab  Excel
Lab  CSV
Flashdrive
Paper sheets
Probe  CSV
So, that’s where AWQMS comes in. It gives you a single place where you can consolidate all your water quality related data, including Beach Notifications and Actions. It provides secure, concurrent access for multiple users. Everyone knows where the data is and there’s no confusion about which “copy” of the data is the latest. It’s your one-stop shop for local data management and analysis.

77. The AWQMS Big Picture Real World Grab S. Habitat As. Probe S.
Fish Census
Benthic M.I.
Census
“Data Limbo”
Lab  .PDF
Lab  Excel
Lab  CSV
Flashdrive
Paper sheets
Probe  CSV
Local Information
AWQMS
All Your Data
Local Analysis Tools
Local Data Verification
Local Data QA/QC
Local Decision Support
Local Data Assessment
Control what is
sent to WQX and
when it is sent
Local Portal (Optnl.)
(Click through this slide)

78. The AWQMS Big Picture Public Information WQX/STORET Water Quality
Portal (WQP)
Only Public Data
Public Analysis
Tools
Real World
Grab S.
Habitat As.
Probe S.
Fish Census
Benthic M.I.
Census
“Data Limbo”
Lab  .PDF
Lab  Excel
Lab  CSV
Flashdrive
Paper sheets
Probe  CSV
Local Information
Local AWQMS
All Your Data
Local Analysis Tools
Local Data Verification
Local Data QA/QC
Local Decision Support
Local Data Assessment
Control what is
sent to WQX and
when it is sent
Local Portal (Optnl.)
When your data is declared to be finalized, AWQMS gives you a mechanism to flag data as “ready to send” to WQX for public use and access to the data. WQX, the STORET Data Warehouse, and the new Water Quality Portal are for public analysis of public data using public tools. Whereas, AWQMS helps you make sure your data is ready to go to WQX BEFORE the data is made public.

79. The AWQMS Big Picture Real World Grab S. Habitat As. Probe S.
Fish Census
Benthic M.I.
Census
“Data Limbo”
Lab  .PDF
Lab  Excel
Lab  CSV
Flashdrive
Paper sheets
Probe  CSV
Local Information
AWQMS
All Your Data
Local Analysis Tools
Local Data Verification
Local Data QA/QC
Local Decision Support
Local Data Assessment
Control what is
sent to WQX and
when it is sent
Local Portal (Optnl.)
Public Information
WQX/STORET
Water Quality
Portal (WQP)
Only Public Data
Public Analysis
Tools