QA/QC and QAPP: How to Get Professional Quality Data From a Volunteer Program

Cook Inlet Keeper  CEMP program volunteers, 1906 observations Effectiveness of CEMP report Community-based lab  CEMP program volunteers, 1906 observations Effectiveness of CEMP report Community-based lab

The Cook Inlet watershed covering 47,000 square miles of Southcentral Alaska. Melting snow and ice from mount McKinley, the Chugach Mountains and the Aleutian Range drains into rivers such as the mighty Susitna, Matanuska and Kenai, which feed the productive waters of Cook Inlet.

G Goals and Objectives: G Inventory baseline water quality in the waters of Cook Inlet Basin. G Detect and report significant changes and track water quality trends. G Raise public awareness of the importance of water quality through hands on involvement.

Great! We know our data is scientifically defensible. But is our data used? And how is it used? Great! We know our data is scientifically defensible. But is our data used? And how is it used?

Federal Government (Ex: Clean Water Act) State designation and standards Scientific Community

Assessing Water Quality is a huge task requiring large amounts of data … and we have very limited resources (both financial/human)

Scientist vs. Volunteer Due to lack of data, you would think data would be in high demand and volunteer programs would be supported to supplement national efforts to monitor water quality.  Volunteer monitoring programs have potential to both provide data and help to improve quality of water with outreach and education. Due to lack of data, you would think data would be in high demand and volunteer programs would be supported to supplement national efforts to monitor water quality.  Volunteer monitoring programs have potential to both provide data and help to improve quality of water with outreach and education.

Dilemma Agencies want/need Water Quality Data and the lack of funding and staff time makes it impossible for them to collect it…..  Scientific community skeptical about usefulness of data by citizen monitors.  “Data credibility is the biggest threat to the future of volunteer monitoring programs” -Fatima Paiva Agencies want/need Water Quality Data and the lack of funding and staff time makes it impossible for them to collect it…..  Scientific community skeptical about usefulness of data by citizen monitors.  “Data credibility is the biggest threat to the future of volunteer monitoring programs” -Fatima Paiva

Scientist vs. Volunteer examples  IOWATER  Minnesota  Texas  Missouri  Alaska  New Hampshire  Volunteer Monitor Newsletter  IOWATER  Minnesota  Texas  Missouri  Alaska  New Hampshire  Volunteer Monitor Newsletter

Figure 2. Comparison of citizen monitoring data to professional ambient monitoring data. Preliminary findings indicate a good match in overall trends, with similar testing results for dissolved oxygen, pH, temperature and nitrate concentrations. These graphs show average monthly concentrations.

Successful programs all had the following key ingredients: Strong Leadership Partnerships with identified local data users Informed, involved, motivated volunteers Successful programs all had the following key ingredients: Strong Leadership Partnerships with identified local data users Informed, involved, motivated volunteers

Most common issues…  Volunteer monitoring groups’ data are frequently not used by their intended users due to:  Difficulty in measuring long-term changes in complex ecosystems  Lack of monitoring plans/QAPP  Lack of communication.  Groups find it hard to move from data collection to next step.  Volunteer monitoring groups’ data are frequently not used by their intended users due to:  Difficulty in measuring long-term changes in complex ecosystems  Lack of monitoring plans/QAPP  Lack of communication.  Groups find it hard to move from data collection to next step.

EPA guidelines for credible monitoring programs:  Pilot Program  Expand Program  Maintain Volunteer Interest/Motivation  Prepare QAAP EPA guidelines for credible monitoring programs:  Pilot Program  Expand Program  Maintain Volunteer Interest/Motivation  Prepare QAAP

At the minimum, your program should: Develop methods to insure data is valid and useful. Allow methods to be scrutinized by scientific community to help improve and legitimize the efforts of volunteer organizations and to determine if data collected by organizations is useful to policy makers and the scientific community. Develop methods to insure data is valid and useful. Allow methods to be scrutinized by scientific community to help improve and legitimize the efforts of volunteer organizations and to determine if data collected by organizations is useful to policy makers and the scientific community.

Monitoring plans  Data on field and lab sheets must go through many steps before being able to be involved in management decisions and actions.  Monitoring plans are written documents that answer strategic questions designed to focus monitoring programs.  Data on field and lab sheets must go through many steps before being able to be involved in management decisions and actions.  Monitoring plans are written documents that answer strategic questions designed to focus monitoring programs.

Monitoring Plan

Data Pathway

PREPARE QAAP  Project description  Project organization/responsibilities  Quality assurance objectives  Sampling procedures  Sample custody  Calibration procedures/frequency  Analytical procedures  Data reduction/validation/reporting  Internal quality control checks  Performance and system audits  Preventative maintenance  Specific routine procedures used to assess data precision, accuracy, completeness  Corrective action  Quality assurance reports  Project description  Project organization/responsibilities  Quality assurance objectives  Sampling procedures  Sample custody  Calibration procedures/frequency  Analytical procedures  Data reduction/validation/reporting  Internal quality control checks  Performance and system audits  Preventative maintenance  Specific routine procedures used to assess data precision, accuracy, completeness  Corrective action  Quality assurance reports

What is a QAPP  Written document that describes all aspects of program and detailed QA/QC used to assure data quality.  Describes the organization of the program including SOPs for field and lab.  Written document that describes all aspects of program and detailed QA/QC used to assure data quality.  Describes the organization of the program including SOPs for field and lab.

QA/QC  Demonstrate accuracy and precision of monitoring.  QA-broad plan for maintaining quality in all aspects of program, including quality control measures, sample collection, sample analysis, data management, documentation.  QC-the steps, including measurements, calibrations, etc that assures quality of specific sampling and analytical procedures.  Demonstrate accuracy and precision of monitoring.  QA-broad plan for maintaining quality in all aspects of program, including quality control measures, sample collection, sample analysis, data management, documentation.  QC-the steps, including measurements, calibrations, etc that assures quality of specific sampling and analytical procedures.

QA/QC: Methods of Testing  Extensive training/Established procedures  Firm Data Quality Objectives-sensitivity of method  Repetition  Multiple methods/indicators for testing parameters  Split Sample  Blinds/standards  Regular recertification  Well-maintained database  Knowledgeable staff  Extensive training/Established procedures  Firm Data Quality Objectives-sensitivity of method  Repetition  Multiple methods/indicators for testing parameters  Split Sample  Blinds/standards  Regular recertification  Well-maintained database  Knowledgeable staff

Citizen Success

Statistics  Generally descriptive  Distributions and histograms  T-test and regressions  Generally descriptive  Distributions and histograms  T-test and regressions