Common Issues With Water Quality Submissions New Directions in Data Management Continuous Monitoring Water Quality Assessments Tribal/EPA CWA Training April 27-28, 2016 Mark LeBaron – Gold Systems

Background on Gold Systems Involved with support of STORET for many years in multiple ways Wrote the Water Quality eXchange (WQX) for EPA Wrote the EPA’s online submission tool: “WQX Web” Provide various types of support for WQX and WQX Web Created the Ambient Water Quality Monitoring System (AWQMS) Work with many tribes in EPA Regions 10, 9, 8, 7, 6 & 5

What is AWQMS? A web-based water quality sample and observations database application Consolidate Validate Analyze Assess Share

What is AWQMS? Community-based development approach Appx. 70 tribes, 5 States, Appx. 80 watershed groups Appx. 140M Result Records In WQX. AWQMS = 33M Or 24% At WQX!! (as of 2/8/2016) Built-in EPA Exchange Network Node Client Validates All WQX Data Rules Auto-synchronizes pick-list values from WQX

What is AWQMS? Configurable Batch Import Configurable Manual Data Entry QA/QC Data: –During Batch Import –During Manual Data Entry –After The Fact Systems Integration via RESTful and other Web Services Public Water Quality Portal (If Needed)

What is AWQMS? Data Analysis Tools: –Graphs, –Statistical Reports, –Mapping, –Data Exports

New AWQMS Features Under Way Improved Continuous Monitoring Support Improved 106 Reporting Support SP 14b Reporting Support GIS Web Services Support Virtual Node Publishing Services Support Improved Support for Assessments

New AWQMS Features Under Way Beach Notifications (for States) Auto-Generation of Metrics Based on Macroinvertebrate Counts Search for Locations (and Associated Activities and Results) by Assessment Unit or by Watershed Management Unit Auto-Generation of metals thresholds

AWQMS is Sustainable Major Current Community Efforts Maryland Department of the Environment Eight Northern Indian Pueblos Council Region 5 Wild Rice Consortium (via Fond du Lac) EPA Region 8

AWQMS is Sustainable Major Current Community Efforts Coalition of Oklahoma AWQMS Tribes (COAT) – via Otoe Missouria Tribe Native American Environmental Protection Coalition (NAEPC) Kickapoo Tribe in Kansas Bishop Paiute Tribe

Common WQX Data Problems (and Potential Solutions)

Common WQX Data Problems Lack of data management processes and mechanisms Files on multiple laptops or flash drives Files in multiple formats No backups Who’s got the master files? Which files have been submitted to WQX? Etc. etc.

Common WQX Data Problems Lack of data management processes and mechanisms - Solution Get or build a database management system. “Don’t bring a spreadsheet to a database fight”

Common WQX Data Problems Monitoring Location Confusion Multiple variants for a single location’s ID. Essentially the same location identified differently by multiple people or groups “Placeholder” monitoring locations created to house trip/field blank QA/QC data.

Common WQX Data Problems Monitoring Location Confusion - Solutions Enforce usage of allowed value tables (picklists). Search existing locations list prior to addition of a new one Use of WQX monitoring location “Alias” fields Use QA/QC WQX Activity type.

Common WQX Data Problems Lab Data Missing Required Metadata (or other Lab Output Issues) Paper or.pdf rather than text file or spreadsheet Missing analytical method information Missing sample fraction information Collection or other dates/times combined as a single data element. Unnecessary white space in the data file Detection condition issues

Common WQX Data Problems Lab Data Missing Required Metadata (or other Lab Output Issues) - Solutions Put it in your agreement with the lab Electronic files: text or spreadsheet Provide them an example output file Have them split Analytical Method into: Source (e.g. USEPA, APHA, etc.) and Method Number Provide them the list of WQX Sample Fractions and definitions

Common WQX Data Problems Lab Data Missing Required Metadata (or other Lab Output Issues) - Solutions Put it in your agreement with the lab Have them split Sample Collection and Analysis dates/times into separate columns for date and time. Show them: no unnecessary white space in the data file. Just data! Have a separate column for Detection Condition (and leave the result value column empty)

Continuous Data Management

Current AWQMS Support of CM Data: “Meh” Database Optimized for Discrete Sample Data Data Analysis Tools Optimized for Discrete Sample Data Continuous data does not “flow” to EN; only summary data

Continuous Data Submission Today Data User Sensor Organization WQX STORET WQ Portal metadata Summary Data Raw data file attachment metaata Summary data

Data EPA Direction (As We Understand it) Data User Sensor Organization WQX Metadata (Monitoring Locations, etc.) Raw Data Discovery Tool (AKA: “Sensor Catalog” “Data Appliance” Raw Data CWQSD Attributes CS-W Service Raw Data

AWQMS Plan to Improve Support of CM Data Integrate Discrete and CM data for: Single Environment = Reduced Training and Less Work Data Analysis – Include and compare both discrete and CM data Integration w/EPA’s Planned Solution = Greater Availability of Data

AWQMS Becomes a “Sensor Data Appliance” Data Data User Sensor Organization WQX Raw Data Discovery Tool (AKA: “Sensor Catalog” “Data Appliance” Raw Data CWQSD Attributes CS-W Service AWQMS (Data Appliance) Metadata (Monitoring Locations, etc.) Raw Data

Water Quality Assessments

Current AWQMS Support: “Meh.” Can find data appropriate for assessment Can import Assessment Units and Associated Parameter “Uses” and Standards Exceedance Summary Export Exceedance Details Export Actual assessment takes place outside of AWQMS No assessment “flow” from AWQMS to EPA EN

Water Quality Assessments Today

Water Quality Assessments EPA Direction

AWQMS Plan to Improve Support of Water Quality Assessments Improve Assessment meta-data management capabilities in AWQMS: –Assessment units & monitoring location associations –Related Designated uses, –Related water quality standards

AWQMS Plan to Improve Support of Water Quality Assessments Provide data analysis tools (graphs, reports, and mapping tools) in AWQMS optimized for assessment purposes Provide ADB-like functionality for capturing assessments.

AWQMS Plan to Improve Support of Water Quality Assessments Provide tools for associating assessments to the underlying raw data and/or analysis Add ability to “flow” ATTAINS data via the new ATTAINS Exchange Network Services

AWQMS Direction

Measures WQ-SP14a and b

Support for Measures WQ-SP14a and b SP14a – Identify monitoring stations where waters are improving Techically, AWQMS already supports SP14a. SP14b - Identify monitoring stations on tribal lands where the waters are meeting tribal water quality objectives. Additional functionality needed to better support SP14b “Path A” (statistical analysis)

Support for Measures WQ-SP14a and b Much of what is needed is in place to do this in AWQMS. Need to assist tribes to populate the water quality objectives tables in AWQMS Need to provide improved AWQMS navigation to guide user to existing functionality

Support for Measure WQ-SP14b (Continued) Add reports/graphs to evaluate the locations. Add ability to record the evaluation per monitoring location.

Thank you!

Continuous Monitoring Data Improved Support Data Management & Storage Data Analysis Submission to the EPA Hooked in to national solution

Using AWQMS to Store and Analyze Drinking Water Data A “home” for tribal drinking water data Import, Store, Manage Data Analyze Data Flagged so it won’t flow to WQX

Tribal Waters Use-Assessment Tracking Set up standards per designated use Group monitoring stations into “Assessment Units” Evaluate stations and assessment units against standards (per designated use) Log assessments Submit assessment data In line with the national solution

Auto-Generation of Metrics Based on Macroinvertebrate Counts Currently can record metrics but have to calculate them external to AWQMS Enhancement will add a limited set of abundance type metrics Necessitates addition of an ability to configure a “profile” per taxon

Improved Analysis and Reporting of Biological Data Current tools were developed for chemical/physical parameters Not designed for “counts” Not designed for metrics Enhancement will resolve these issues

On-The-Fly Water Quality Standards Generation for Metals Parameters The “standard” for certain types of parameters depends somewhat on the context at the time of the sample (e.g. the hardness). Configure groups of such parameters For a limited list of parameters, auto-calculate and store the standard on-the-fly based on other parameter results captured

On-The-Fly Water Quality Standards Generation for Metals Parameters (Continued) Improve data analysis tools to compare parameter results against the calculated standard