From STORET to WQX New Jersey’s Solution for Integrating Ambient Water Quality, Facility and GIS Information from Federal, State, County & Volunteer Organizations Harry Chen New Jersey Department of Environmental Protection November 28, 2007 Austin, TX

Project Purpose Integrate water quality data Integrate water quality data Streamline and simplify data transfer Streamline and simplify data transfer Provide better, more timely access to water quality data Provide better, more timely access to water quality data –Aid decision-making –Improve resource efficiency

Goals Develop an up to date, searchable inventory of ambient water quality monitoring in New Jersey Develop an up to date, searchable inventory of ambient water quality monitoring in New Jersey Integrate ambient water quality monitoring data into NJDEP’s enterprise environmental data management system to enable more integrated environmental decisions Integrate ambient water quality monitoring data into NJDEP’s enterprise environmental data management system to enable more integrated environmental decisions Make NJ ambient water quality monitoring data available to the public Make NJ ambient water quality monitoring data available to the public Report ambient water quality monitoring data to USEPA Report ambient water quality monitoring data to USEPA

Project Partners NJDEP NJDEP –Bureau of Freshwater and Biological Monitoring –Bureau of Marine Water Monitoring –Bureau of Water Quality Standards and Assessment –Division of Watershed Management –Division of Science, Research, and Technology USGS USGS Department of Health and Senior Services (DHSS) Department of Health and Senior Services (DHSS) County Governments County Governments Volunteer Monitoring Organizations Volunteer Monitoring Organizations NJ Water Monitoring Coordinating Council (Inventory) NJ Water Monitoring Coordinating Council (Inventory)

Existing Ambient Water Quality Data Producers

Field For m Bench Sheet AMNET Field/Hab dB AMNET Field/Hab dB AMNET Taxa dB AMNET Taxa dB Reformat Field/Habitat dB Reformat Field/Habitat dB FTP Oracle Export NJ STORET EPA Migrate Delete Migration Field For m FIBI dB Extensive Manual Reformat Of Text File DHSS Chem Results QC Labs Chem Results Field Form Lakes dB Reformat Lakes dB DHSS Chem Results QC Labs Chem Results Field Form Others Reformat Taxa dB Reformat Taxa dB Keypunch Field Msr Habitat Taxa counts Sample Info Keypunch Field Msr Habitat Sample Info Keypunch Taxa counts Import e-file Chem Results Import e-file Chem Results Keypunch Field Msr Habitat Sample Info Proofreading dB ODBC Connect Manual Proofreading Reports Corrections Corrections Text file Text file Make Final dB Change Result Status To Final Text file Keypunch Chem Results Keypunch Keypunch Field Msr Sample Info Text file MS Access Link Others Other Projects dBs Field Staff Data Staff Existing NJDEP Data Entry Mechanisms Note: Proofreading and correction steps occur for all data, but are not shown due to space limitations. Sample Tracking dB ODBC Connect

Water Quality Data Management Model

Core project components WQX data flow, consisting of the following WQX data flow, consisting of the following –Data-in: E2 WQX module with data validation –Data transmission to and from COMPASS –Data-out: NJ Water Monitoring Inventory / Data Portal Outbound data transmission to EPA CDX Outbound data transmission to EPA CDX

High-Level Data Flow EPA CDX E2COMPASS ENDS

Data System Integration E2 E2 COMPASS COMPASS ENDS ENDS Data will be transmitted through each component using NJ’s Exchange Network NODE.

What is E2? The Electronic Environmental Reporting system (E2) is a secured XML/Internet based system.

What is COMPASS? Comprehensive Organizational Monitoring Program and Assessment Support System (COMPASS) is an enterprise-wide methodology and information management solution.

What is ENDS? Exchange Network Data Services are web- services based applications used to exchange data between EN-compliant Nodes.

Future Ambient Water Quality Data Producers EPA & USGS Web Services 4

Data Flow for Submission of “All Publicly Available Data” From Multiple Organizations Ex. 305(b) and 303(d) Assessments Sampling Data

Data Flow When An Organization Uses Multiple Groups to Generate Results (Ex. Field Measurements by One Group and Lab Analysis by a Second Group)

Data Flow From a Single Organization (Ex. Field and Lab Work Done by Same Group)

Data Submission Options

CIMP CSV-file Import Mapping Procedures

Data Field (column) Mapping

Data Normalization

Mapping to Domain values

Save Mapping configurations

Water Quality Data Transmission to CDX

GIS Integration Link Monitoring InventoryData Reports Link Web-based Maps

GIS Proposals: EN Browser hosted by ECOS, a national solution EN Browser hosted by ECOS, a national solution NJ-GeoWeb, a NJ solution NJ-GeoWeb, a NJ solution

EN Browser Web services data discovery tool –Allows users to discover and execute Data Requests on Exchange Network Nodes –Offers users the ability to remove (filter) unwanted sections of the XML data set returned from a Data Service –Users have the option of choosing the format that they would like to retrieve the data in –Can integrate with enterprise data reporting tools

NJ-GeoWeb A New Vision for Seeking Spatial Intelligence. Integrating customized GIS and analysis tools with enterprise databases for improved decision-making on the Internet.

What is: NJ-GeoWeb DEP’s next generation i-MapNJ on the Interent. Replaces ArcIntelligence on the intranet. DEP’s next generation i-MapNJ on the Interent. Replaces ArcIntelligence on the intranet. Uses profiles (tabs) that default data layers and reports (ie. Child Care, What’s in My Neighborhood, Smart Growth, Business Viewer…) Uses profiles (tabs) that default data layers and reports (ie. Child Care, What’s in My Neighborhood, Smart Growth, Business Viewer…) Allows Internet users to view, interact with, analyze and submit points, lines and polygons Allows Internet users to view, interact with, analyze and submit points, lines and polygons Links to departmental databases (DataMiner, eNJEMS, COMPASS..). Links to departmental databases (DataMiner, eNJEMS, COMPASS..). Uses “enterprise” data from all Agencies (and serves from the Web) Uses “enterprise” data from all Agencies (and serves from the Web) Links to external services on the web. Links to external services on the web. Working with NJ OIT make NJ-GeoWeb a state agency resource. Working with NJ OIT make NJ-GeoWeb a state agency resource.

Sustainable Growth Align Economic and Env. Goals Env. Sensitive Areas Green Technologies Data Server Data Server SurveillanceInspectionsePermits PermittingComplianceEnforcement Water/Sewer Infrastructure Water Allocation Storm Water Runoff Monitoring Taking care of Business Clean & Plentiful Water Regulated Sites Incident Mgmt Flood Mapping Endangered Species What’s in My Backyard Economic Incentives Employment Info Revitalize and Redevelop SDE “NJ-GeoWeb” Data Live Feeds from State Government L.awrence Thornton, NJDEP

WQDE Project Challenges Pulling data from USGS (web services in development) Pulling data from USGS (web services in development) Pulling data from EPA (web services in development) Pulling data from EPA (web services in development) Coordinating with a large number of project partners Coordinating with a large number of project partners –Requires compromise Re-engineering current reporting procedures, while designing new WQX reporting procedures Re-engineering current reporting procedures, while designing new WQX reporting procedures –How to teach an old dog new tricks?

Status of NJDEP Data Exchanges Water Quality Data Exchange Water Quality Data Exchange –Completed Business Process Analysis with 11 project partners –Identified Needs and Analyzed Options for implementing data exchange –Completed Requirement Specification –Design currently in progress Volunteer Monitoring Volunteer Monitoring –Application in production –Currently designing reports

Business Process Analysis Business Process Analysis Functional Requirements Specification and System Design Functional Requirements Specification and System Design System Development, Data Preparation and Implementation System Development, Data Preparation and Implementation Acceptance Testing Acceptance Testing User Training, Documentation and Outreach User Training, Documentation and Outreach Project Phases and Tasks:

Next Steps Complete System Design Complete System Design Programming and Development Programming and Development User Acceptance Testing User Acceptance Testing Documentation and Training Documentation and Training Production Deployment Production Deployment Send data to EPA Send data to EPA Target Date: October 2008 Target Date: October 2008

Thanks! Contact Info: NJDEP- Office of Info Resources Management Name: Harry Chen Phone: (609)