Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Quality Assurance and Data Quality for Stormwater Programs Richard Smith INCOG June 27, 2006 8 th Annual EPA Region 6 MS4 Stormwater Conference.

Similar presentations


Presentation on theme: "1 Quality Assurance and Data Quality for Stormwater Programs Richard Smith INCOG June 27, 2006 8 th Annual EPA Region 6 MS4 Stormwater Conference."— Presentation transcript:

1 1 Quality Assurance and Data Quality for Stormwater Programs Richard Smith INCOG June 27, th Annual EPA Region 6 MS4 Stormwater Conference

2 2 Data Quality and Quality Assurance “QA” is an essential component of creating and maintaining the usefulness of data and information. “QA” is an essential component of creating and maintaining the usefulness of data and information. QA addresses : QA addresses : “Monitoring” data (e.g. field and lab measurement data) “Monitoring” data (e.g. field and lab measurement data) Observation data (e.g. visual inspections) Observation data (e.g. visual inspections) Records of conversations, meetings, s, letters, etc. Records of conversations, meetings, s, letters, etc. Data summarization and analyses (e.g. statistics, estimates) Data summarization and analyses (e.g. statistics, estimates) Reports and data transfers (e.g. spreadsheets, letters) Reports and data transfers (e.g. spreadsheets, letters) In-house processes for communication and record-keeping In-house processes for communication and record-keeping Storage, calibration and use of equipment and supplies Storage, calibration and use of equipment and supplies “QA = CYA”

3 3 How Will Your Data Be Used ? Permitting Authority – (Annual Report / DMRs / Phase II audits / P.A. pollution & compliance enforcement). Permitting Authority – (Annual Report / DMRs / Phase II audits / P.A. pollution & compliance enforcement). Other State and Federal Agencies – to fulfill their own data collection efforts for projects / databases. Other State and Federal Agencies – to fulfill their own data collection efforts for projects / databases. City Councils / County Commissions – program updates, budgets, justify additional resources or changes to ordinances, etc. City Councils / County Commissions – program updates, budgets, justify additional resources or changes to ordinances, etc. Internal Staff Requests – information flow within city. Internal Staff Requests – information flow within city. Citizens / Local Parties – public education, data requests. Citizens / Local Parties – public education, data requests. Local Enforcement Actions – building your case. Local Enforcement Actions – building your case. Local Media – radio, newspaper, television. Local Media – radio, newspaper, television. The Data Needs will dictate how you collect, store and report data.

4 4 Budget items Photographs Phase II Data Streaming Data Management System Lab & Field Data Visual Observations Inspection Interviews Data from Agencies Management Reports Reports to City Staff Reports to Citizens DATA IN-FLOWSDATA OUT-FLOWS Data to Agencies ODEQ Annual Report Program Audits Complaint Tracking Who, what, where, when, why, how BMP Quantities Training Records Map Data Reports to Councils Data for Court Information to Media Enforcement Data

5 5 MS4 / FACILITY INSPECTIONS CONSTRUCTION INSPECTIONS EDUCATION & TRAINING DATA MANAGER & REPORTING CITY MANAGER PUBLIC WORKS DIR. CITY COUNCIL STORMWATER PROGRAM MANAGER CONSULTANTS AGENCIES & SUBSTATES LOCAL CITIZENS & BUSINESSES VENDORS Stormwater Program Manager Many cities and counties struggle to define a management structure.

6 6 Map data GIS, GPS, AutoCAD, paper. Map data GIS, GPS, AutoCAD, paper. QA methods What’s required, most efficient. QA methods What’s required, most efficient. Electronic data sets Tier II, RCRA, TRI, etc. Electronic data sets Tier II, RCRA, TRI, etc. Field test kits D.O. pH, Cond., etc. Field test kits D.O. pH, Cond., etc. Other field gear Samplers, bottles, etc. Other field gear Samplers, bottles, etc. Data Management: Data Management: Software for supervision, records, reports Software for supervision, records, reports Forms for inspections, contact records. etc. Forms for inspections, contact records. etc. Data and monitoring Quality Assurance (QA) Data and monitoring Quality Assurance (QA) Sampling and Lab services Sampling and Lab services Most Phase IIs Need Help With...

7 7 Need for Data Management Paper Filing System Computer Software: Spreadsheet or Database Commercial Vendor Software Customized / Consultant Developed Data Management Options Consider Your Ability to: Purchase Learn Use Maintain

8 8 Many circumstances require verification of information: Many circumstances require verification of information: Measurement data for court cases or enforcement. Measurement data for court cases or enforcement. Reports to City Councils, Commissions, citizens, contractors, consultants, agencies and in-house use. Reports to City Councils, Commissions, citizens, contractors, consultants, agencies and in-house use. Regulatory decision-making (e.g. violations of WQS). Regulatory decision-making (e.g. violations of WQS). Political fallout if your data is discredited. Political fallout if your data is discredited. Formal documentation of QA procedures is wise. Formal documentation of QA procedures is wise. Don’t expect to have a fully developed QA program quickly – give yourself time to learn and experiment with what works best for your local needs. Don’t expect to have a fully developed QA program quickly – give yourself time to learn and experiment with what works best for your local needs. Why is QA Important ?

9 9 To What Activities Should QA Apply ? Technically, all activities pertaining to any type of data collection, analysis, manipulation and reporting.... Technically, all activities pertaining to any type of data collection, analysis, manipulation and reporting.... Determine your highest priorities first. For example: Determine your highest priorities first. For example: Visual inspections / lite-monitoring of MS4 Visual inspections / lite-monitoring of MS4 Source tracking inspections Source tracking inspections Construction site inspections Construction site inspections If private consultants and/or labs are used, be sure that they employ appropriate and adequate QA to their activities. If private consultants and/or labs are used, be sure that they employ appropriate and adequate QA to their activities. Make this part of your contract specifications, especially if their work is needed for enforcement or court evidence. Make this part of your contract specifications, especially if their work is needed for enforcement or court evidence.

10 10 Even Simple Test Kits Need Attention Expired chemicals - field test kits have chemicals that expire – keep records of refresh purchases. Expired chemicals - field test kits have chemicals that expire – keep records of refresh purchases. Some kits require “calibration” – use fresh calibration standards and keep a log of calibration activities. Some kits require “calibration” – use fresh calibration standards and keep a log of calibration activities. Some kits can measure “check samples” (of known analyte concentration) – perform checks frequently and keep records of all tests. Some kits can measure “check samples” (of known analyte concentration) – perform checks frequently and keep records of all tests. SOPs help ensure continuous proper use, and aid in training new employees. SOPs help ensure continuous proper use, and aid in training new employees. Maintenance includes verifying equipment function and replacing defective parts before re-use of the kit. Detailed maintenance records should be kept. Maintenance includes verifying equipment function and replacing defective parts before re-use of the kit. Detailed maintenance records should be kept.

11 11 Types of Field Instruments Comparator Test Kits Test Strips Titrations Colorimeters Multi-Test Kits These types of kits rely upon chemical reactions to indicate the presence and/or concentration of one specific parameter. No endorsement of products is intended. An SOP should be written for each type of kit.

12 12 Types of Field Instruments Test Pens Single Parameter Meters Multi-Parameter Meters Field Fluorometer Turbidimeter Flow Meter These types of kits measure directly the concentration of one or more parameters. No endorsement of products is intended. An SOP should be written for each device and procedure.

13 13 QA Concepts Precision is a measure of how close multiple readings are of the same sample. Accuracy is a measure of how close a reading is to the known value. Both are important concepts in assessing quality of data. Both are normally reported with laboratory data.

14 14 More QA Concepts Representativeness Completeness Comparability Chain of Custody Sample Labels Does the sample represent what you intend? Do you have enough samples? Are sampling methods useable to others? Legal documentation of “ownership” of samples during all phases of transport. Adequate information on each sample bottle to show ownership, collection information and intended analyses.

15 15 Types of QA Documentation to Consider QAPP – QA Project Plan - how you conduct data gathering, analysis and reporting for your Phase II program. QAPP – QA Project Plan - how you conduct data gathering, analysis and reporting for your Phase II program. DQOs – Data Quality Objectives – deciding in the beginning what types of data you will be collecting and, most important, how it will be used. DQOs – Data Quality Objectives – deciding in the beginning what types of data you will be collecting and, most important, how it will be used. QMP – Quality Management Plan – down the road, you may consider this – describes your overall data management and communication system. QMP – Quality Management Plan – down the road, you may consider this – describes your overall data management and communication system. SOPs – Standard Operating Procedures – detailed instructions for how you perform specific data collection activities, including use, storage and care of measurement equipment. SOPs – Standard Operating Procedures – detailed instructions for how you perform specific data collection activities, including use, storage and care of measurement equipment.

16 16 Each State’s General Permit should provide specific requirements. Each State’s General Permit should provide specific requirements. In Oklahoma, OKR04 Part V lists a number of requirements for “Monitoring, Record Keeping and Reporting” In Oklahoma, OKR04 Part V lists a number of requirements for “Monitoring, Record Keeping and Reporting” “Samples and measurements...shall be representative...” “Samples and measurements...shall be representative...” “If lab analysis is conducted it must [use] 40 CFR Part 136” [methods] “If lab analysis is conducted it must [use] 40 CFR Part 136” [methods] “Monitoring records must include...[names, dates, times, place, methods, etc.]” “Monitoring records must include...[names, dates, times, place, methods, etc.]” “Retain records of all monitoring information” (e.g. copies of reports, instrument charts, etc.) “Retain records of all monitoring information” (e.g. copies of reports, instrument charts, etc.) What Is Absolutely Required ?

17 17 Are Formal QA Documents Required ? Your State’s General Permit will state which, if any, QA documents will be required (e.g. QAPP, DQOs, QMP and/or SOPs). Your State’s General Permit will state which, if any, QA documents will be required (e.g. QAPP, DQOs, QMP and/or SOPs). In Oklahoma, OKR04 does not specifically require any of these documents. In Oklahoma, OKR04 does not specifically require any of these documents. However, one must consider the “CYA” aspect of QA as well as what the minimum permit requirements are. However, one must consider the “CYA” aspect of QA as well as what the minimum permit requirements are. Particularly if your data are to be used in court, for local enforcement or for making regulatory decisions... Particularly if your data are to be used in court, for local enforcement or for making regulatory decisions... Or for that matter, just giving yourself credibility to your manager, peers, mayor and citizens. Or for that matter, just giving yourself credibility to your manager, peers, mayor and citizens.

18 18 How to Start a Formal QA Program Managers, Mayors and Councilmen must learn the importance of good QA’d data, and how it will help them defend their programs. Managers, Mayors and Councilmen must learn the importance of good QA’d data, and how it will help them defend their programs. Don’t try to do it all at once – expect changes as you go. Don’t try to do it all at once – expect changes as you go. Decide what level of QA you need to apply to each type of data and activity. Decide what level of QA you need to apply to each type of data and activity. Start with SOPs and a basic QAPP. Start with SOPs and a basic QAPP. Seek guidance from experienced resources. Seek guidance from experienced resources. Be comfortable with what you create – stick to basics. Be comfortable with what you create – stick to basics. Amend as your Phase II program grows – QA is dynamic. Amend as your Phase II program grows – QA is dynamic.

19 19 A Good Beginning QA Program Start applying QA to “hard data” first – e.g., chemical data from field measurements, field test kits and lab data as well as visual observation data from visual inspections. Start applying QA to “hard data” first – e.g., chemical data from field measurements, field test kits and lab data as well as visual observation data from visual inspections. DQOs - Decide upon the level of data quality you want for the main types of data you will be collecting. Use of data. DQOs - Decide upon the level of data quality you want for the main types of data you will be collecting. Use of data. SOPs – identify all routinely repeatable actions and write detailed step by step instructions and list supplies needed. SOPs – identify all routinely repeatable actions and write detailed step by step instructions and list supplies needed. QAPP – prepare an umbrella QA plan for all activities requiring formal QA attention. QAPP – prepare an umbrella QA plan for all activities requiring formal QA attention. Lab and Service contracts – include in your contract (and QAPP) what technical requirements you want for lab services and from vendors and consultants that will be performing technical services (e.g., inspections). Lab and Service contracts – include in your contract (and QAPP) what technical requirements you want for lab services and from vendors and consultants that will be performing technical services (e.g., inspections).

20 20 EPA guidance documents: EPA guidance documents: DQO Guidance= QA/G-4 DQO Guidance= QA/G-4 QAPP Guidance= QA/R-5 QAPP Guidance= QA/R-5 QMP Guidance= QA/R-2 QMP Guidance= QA/R-2 SOP Guidance= QA/G-6 SOP Guidance= QA/G-6 These are not approvable by EPA unless they pertain to projects funded by EPA water quality grants. These are not approvable by EPA unless they pertain to projects funded by EPA water quality grants. EPA periodically offers QA training courses. EPA periodically offers QA training courses. Contact your Permitting Authority and substates for QA advice. Contact your Permitting Authority and substates for QA advice. EPA’s QA Resources

21 Thank you for your infinite patience. Any Questions ?


Download ppt "1 Quality Assurance and Data Quality for Stormwater Programs Richard Smith INCOG June 27, 2006 8 th Annual EPA Region 6 MS4 Stormwater Conference."

Similar presentations


Ads by Google