1. QA/QC FOR ENVIRONMENTAL MEASUREMENT
Unit 4: Module 13, Lecture 1

2. QA/QC for environmental measurement
What is quality assurance (QA)?
a broad plan for maintaining quality in all aspects of a program
- Keith et al. (1983)
establishes the need for quality control (QC)
Photos: 1.) taken from USGS web site
2.) taken from NRRI – Field Photos
Quality Assurance (QA) has been defined as a broad plan for maintaining quality in all aspects of a program.
QA as it relates to environmental assessment also refers to the overall management system and includes:
Organization
Planning
Data collection
Documentation
Evaluation
and Reporting Activities
All designed to address one purpose – the success of the environmental assessment.
QA is designed to assure that the data gathered and reported on in an environmental assessment meets the defined standards with a stated level of confidence.
QA establishes the need for the implementation of quality control measures

3. QA/QC for environmental measurement
Quality control refers to routine technical activities – the purpose of which is to control error
QC can be considered the “HOW” of the QA process
applicable to field, lab and office procedures
Photos come from: 1.) - group data discussions
2.) - accurate measurement at lab scale
QC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Quality Control refers to the routine technical activities the essential purpose of which is to control error. Errors are possible in the field, laboratory, and the office and so QC is an integral part of each of these functions. QC is generally part of the methods, standards, testing and certification infrastructure outlined by governmental agencies and professional organizations. QC should include both internal (controlled by those implementing the project) and external (controled by those outside of the project) measures (both are defined and discussed in later slides).

4. QA for environmental measurement
Why do we need quality assurance for environmental measurement?
understand data reliability
quantify areas of analytical uncertainty
standardize measurement to allow for repeatable and comparable data across time and place
Why do we need quality assurance in environmental measure?
A) To understand just how reliable our data is and what amount of validity we can put in the measures they provide – things can hardly ever be 100% accurate - (applies to sample collection, analytical analysis, and data evaluation)
B) For the areas we don’t know with 100 % accuracy, we need to give a value (quantify) how much accuracy were are able to assign to the information being evaluated - (includes all aspects from sampling to the evaluation of data results)
C) To standardize measurement and allow for reliable measures that are precise, accurate, repeatable and comparable across time and place (all within an acceptable level of accuracy).
QA ensures that data will meet defined standards of quality with a standard level of confidence.
Data is used to make important decisions and requires the decision maker to have a level of confidence about the reliability of information (data) being used to make those decisions – Historically, regulatory decisions were often made based upon ‘taking someone’s word’ about the reliability of data - but as a society we are becoming more informed and more sophisticated (more skeptical?) in our decision processes. Would you make a decision on data if you weren’t sure it was valid?
QA ensures that data will meet defined standards of quality with a standard level of confidence

5. QA for environmental assessment
Need for consistency - monitoring a copper impacted stream
Is this problem getting worse or is being mitigated?
How can you tell?
Why is standardized measurement so important?
Photo from (Newell, NOAA, 1994) - shows a creek contaminated with copper. The turquoise color is copper precipitate on rocks.
The goal is to make data reproducible over time and place. Data collected for pollution assessments and site mitigation projects is often gathered over time and used comparatively to determine whether a problem may be getting worse or is being mitigated. For this reason standardized measurements are needed to make any type of reasonable comparison.
An example of the need for standardization and consistency - NOAA Case study is outlined below (
BLACKBIRD MINE RESTORATION
Date of incident: Mining activity began in late 1800s, peaked in late 1940s-1950s Location: An inactive mine site in the Panther creek watershed, a tributary of the Salmon River in east central Idaho Trustees: NOAA, State of Idaho and the U.S. Forest Service Source: Inactive mine site Release: Copper and other hazardous materials Primary Injury: Loss of chinook salmon and steelhead habitat Restoration: Reintroduction of chinook salmon and in-kind habitat restoration (estimated cost $5.2 million)
SITE SUMMARY: Snake River spring/summer chinook salmon (listed as threatened under the Endangered Species Act) were eliminated from Panther Creek by the early 1960s due to hazardous substances contamination. Panther Creek is one of the larger tributaries of the Salmon River and historically supported substantial runs of chinook salmon (2,000 adults returning annually). The stream also supported significant numbers of steelhead. In 1993, the Environmental Protection Agency (EPA) proposed the site to its National Priorities List for further investigation and possible cleanup under Superfund authorities. In 1994, EPA initiated a Remedial Investigation/Feasibility Study for site cleanup.
The Settlement: The Consent Decree settling the case requires the parties responsible for the release to restore the water quality in Panther Creek by The responsible parties were also required to fund a program to reintroduce chinook salmon to Panther Creek; implement a Biological Restoration and Compensation Plan (BRCP) to restore, enhance, and create anadromous and resident salmonid habitat in site-impacted and out-of-basin streams; fund trustee oversight of BRCP implementation; and reimburse trustees' past damage assessment costs.

6. Where is QA applicable?
Who in the environmental arena uses quality assurance measures and when is it necessary?
nearly all REGULATORY decisions about: issuing permits, monitoring human health and environmental quality, pollution investigations and the progress of site remediation are based on one thing -- the data gathered from the site.
Quality Assurance (QA) Programs are used by:
Regulatory agencies – regulatory decisions about:
environmental risk,
remediation,
issuing permits,
monitoring surface and ground water quality,
and site investigations
Regulatory decisions related to the above areas of compliance are based on data collected from the site, how it is collected, tested, analyzed, and summarized. The EPA and other regulatory agencies often evaluate how QA goals are met during a project by evaluating the projects Quality Assurance Project Plan (QAPP) we will discuss QAPPS furtherer later on in this presentation.

7. Where is QA applicable?
What other environmental arenas utilize quality assurance measures?
laboratories
universities and research organization
PHOTO From NRRI
Quality Assurance/Quality Control Programs are also used by the following:
Laboratories – whether it is a analytical laboratory testing for individual chemical analytes in soil or groundwater, an engineering lab testing structural parameters of soil, or a lab technician counting the number of E. coli bacteria under a microscope – for any of these analysis to mean something, they must all follow specific methods by which the quality of the analysis being conducted meets at least a minimum standard by which all similar type analysis can be evaluated.
Universities and other Research Programs often use well know and established methods of data collection and analysis to assure the results of their research and study will be accepted as valid. Government established research programs such as the US Geological Survey (USGS) are often responsible for developing and standardizing such QA methods and programs.
Equate the need for QA in the environmental field with the manufacturing industry and needs for specific standards (electrical certifications, drug industry certifications, etc.)

8. Where is QA applicable?
Non-research areas utilize environmental QA measures also
business and industry
lobbying groups and special interests
legislators
lawyers
others
PHOTOS: 1.) 2.)
Quality Assurance/Quality Control Programs are also used by the following:
Businesses and industry – required to meet certain limitations on use and output of a chemical compound or materials with the potential to cause environmental impacts, will use QA standards to assure agency acceptance of data, to provide meaningful results with which to assess compliance, to provide data that would be considered “reliable” in court.
Environmental consulting firms – often working for industry, a public interest groups, etc. use QA for the same reasons – to assure that regulatory agencies will accept the data they present, again to assure that analyzed data meets the quality needs of the project, and to provide “reliable” data to limit potential liability.
Others - Environmental and special interest organizations, lobbying groups, lawyers, etc. – (list as many as your class is able related to the environmental industry note that QA also applies to other industries) - all have a need to assure the quality of data in environmental measurement related to compliance, integrity of presentation, and limiting liability.
Think of some specific examples where accurate data was important to make discussions affecting the public in your area. (e.g.: permitting of chemical out put from a major industry in your area, a recent major lawsuit concerning an environmental outcome - why was QA data important in this case? Recent laws, legislation, city counsel voting, etc. based upon the QA of data presented.)

9. Quality assurance vs. quality control
Quality Assurance (QA)
broad program plan
establishes the need for QC
Quality Controls (QC)
individual checks and balances
the “HOW” of QA
Use this slide as a opportunity to briefly review the meaning of QA and QC – What are the key differences and similarities between the two?
Quality Assurance - a broad plan for maintaining quality in all aspects of a program, also refers to the overall management system and includes: Organization, Planning, Data collection, Documentation, Evaluation, and reporting activities. All designed to address the success of the environmental assessment.
Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. (applies to activities in the field, laboratory, and the office)

10. Where is QC applicable?
Quality control is applicable in all aspects of a project including:
Field data collection and sampling
Laboratory analysis and processing
Data evaluation and assessment
Reporting and project documentation
As we stated earlier… QA ensures that data will meet defined standards of quality with a standard level of confidence.
QC provides the steps by which the same goals are reached.
To be comfortable in the quality of data provided we need to be assured of the processes used in all areas of assessment - from when, where and how the sample was collected; to the processes used by the field technician or laboratory conducting the analysis; to how the data results are interpreted and by whom; to how the information is reported upon and level of accuracy documented.
Without confidence in the data provided all arenas; from regulatory agencies, to research groups, to business officials, to legislators; would have to question many of the decisions they are required to make.
Few agencies will use data unless methods of data collection, storage, and analysis can be documented. Clear and concise documentation of procedures also allows a project to continue over time and space using the same methods. This is particularly important to a project that extends over a long period of time and that intends to establish a baseline of quality information that can be compared over time.
QC provides steps that ensure data will meet defined standards of quality with a standard level of confidence

11. QC in the field QC is particularly critical in filed data collection
often the most costly aspect of a project
data is never reproducible under the exact same condition or setting
Photos from left to right:
1.) provided by NRRI – Collecting sechi disk reading
2.) - USGS water sampling
3.) - logging cores
This slide shows just a very few examples of diverse forms of field data being collected – the class may want to discuss other examples, and why proper sample collection and data analysis is so important – emphasize the reasons provided below.
The first steps to a sound environmental assessment is the evaluation of competent data….and this evaluation begins in the filed. The quality and applicability of the samples and data collected here will guide all other aspects of the environmental evaluation. Data or sample collection from the designated time and place is never reproducible under the exact same conditions and setting. Because of this QA measures must be taken to assure the best possible (most reliable) set of data is obtained. Obtaining data and samples from the filed is also often the most costly aspect of any assessment program to recreate if data is deemed unreliable. These costs are a result of the need for specialized equipment, mobilization to the sample site, the need to redo all other aspects of evaluation, or missing one significant item could render other data useless.
sechi readings
field filtration
logging sea cores

12. QC in the field Standardized field method programs
US Army Corps of Engineers
ASTM methods
USGS classification
APHA
AIHA
NIST
Logos copied from individual web sites as listed below:
Governmental agencies (USGS, EPA, USDHHS, etc.) and professional organizations (ASTM, AIHA, etc. ) have worked together to develop a standards, testing and certification infrastructure to meet the needs of the environmental industry. This slide and the following discussion list a brief overview of many of the standards programs utilized in the industry today:
ASTM – American Society for Testing and Materials - a professional volunteer organization of chemists and engineers - responsible for the development of hundreds of standards related to both filed and laboratory data collection, testing, and analysis. Link to
USGS – US Geologic Survey – US Department of the Interior organization – emphasis on geology, hydrogeology, engineering – responsible for soil classification standards, filed gas measurement standards, in-situ water quality monitoring data collection, etc. Link to
US Army Corps of Engineers – has developed methods for the collection storage and manipulation of materials including sediments used in chemical and toxicological analysis. Link to
APHA – American Public Health Association – Professional “association of individuals and organizations working to improve the public's health and to achieve equity in health status for all.” responsible for most widely accepted standards related to waste water testing, treatment, and assessment. (American Public Health Association. 1995, 1998, Standard methods for the examination of water and wastewater. Washington, D.C., 17th- 19th Editions). Link to
AIHA – American Industrial Hygiene Association – a professional organization of scientists concentrating on occupational and environmental health and safety. – responsible for development of investigations standards in the areas of asbestos, mold, indoor air quality, etc. Link to
NIST - National Institute of Standards and Technologies - is an agency of the U.S. Commerce Department's Technology Administration . NIST Laboratories provide measurements and standards for U.S. industry. Link to

13. QC in the laboratory
Laboratory data analysis, measurement and acquisition:
Chain of custody forms
Equipment calibration
Storage practices
Analytical methods
Holding times
MDLs
Once field data is collected, it must be evaluated with methods and standards that meet the quality requirements of your environmental investigation. Every aspect of analysis needs to be evaluated; form assuring that the samples being analyzed belong to your project (chain of custody), to preventing potential contamination from other sources (storage practices), to assuring the analytical result will be relevant (exp: the equipment is properly calibrated, and testing is conducted within holding times and using proper methods and levels of detection (minimum detection limits - MDLs) sufficient to quantify what it is you are assessing). Again, the goal is to maintain quality in all aspects of the analytical program.

14. QC in the laboratory Standardized laboratory method programs
EPA Methods
State Modified Methods
APHA Methods
ASTM
AIHA
Agency and organization logos were collected from the individual web sites as listed below:
Examples of standards used in laboratory analysis include:
EPA developed methodologies - most commonly used standards for analysis of pollutants and individual chemical analytes also responsible for the auditing and certification of laboratories. Link to
Some state agencies can also certify laboratories and/or have there own modified methods of federal program methods
- Exp: MN Department of Health Modified 465D water analysis list – expands upon the EPA’s standard list of analytes
- Exp: WI Modified DRO and GRO (diesel- and gasoline- range organics) analysis – on both soil and water samples – modifies both field collection for samples, preservatives used, and laboratory analytical requirements.
- List / explain some of the modifications your state or regional offices may use when compared to federal standardized requirements.
APHA – American Public Health Association – Professional “association of individuals and organizations working to improve the public's health and to achieve equity in health status for all.” responsible for most widely accepted standards related to waste water testing, treatment, and assessment. (American Public Health Association. 1995, 1998, Standard methods for the examination of water and wastewater. Washington, D.C., 17th- 19th Editions). Link to
ASTM - professional volunteer organization of chemists and engineers - establishes many methods and standards for assessing the characteristic properties of materials including chemical compounds and liquids (flash point tests, density analysis, etc.) soils (density, permeability, hydraulic conductivity, etc.) and other materials. Link to
AIHA – American Industrial Hygiene Association – a professional organization of scientists concentrating on occupational and environmental health and safety. – responsible for development of laboratory methods and standardized requirements in the areas of asbestos, mold, indoor air quality, etc. Link to

15. QC in the office
In office planning, data evaluation and summation of results:
QAPP – quality assurance
project plan outlines all
project management
assessment and oversight
data validation
mgt_pr.htm
In actuality, although field data collection is very important, the assurance of quality (QA) in an environmental assessment implements QC measures that begin and end in the office (or in the processing stages); initiated with planning, and ending with evaluation of data and conclusions along with project presentation.
If the first steps to a sound environmental assessment is the evaluation of competent data…starting with its proper collection in the field…the first steps to assuring that field collected data will be competent start in the office, or on the drawing board, with the creation of a Quality Assurance Project Plan (QAPP) (discussed in more detail in future slides and lesson 13B).
A quality assurance project plan (QAPP) determines how QA data is collected and what QC measures will be used. The QAPP is a guide to conducting all aspects of an investigation and is originally outlined, developed, and compiled in the office. The QAPP is an invaluable planning and operating tool that should be developed in the early stages of an environmental assessment. In addition to development of the QAPP, other QC measures included in the office aspect of a project involve:
Project management – who will work on the project, what are their qualifications and training? Who will supervises and make sure things progress on schedule and on budget?
Assessment and oversight – who will evaluate the project work as it progresses? Who will take responsibility for the final outcomes?
Data validation – What statistical methods will be used to determine if the data is accurate and /or relevant?
Photo from MN DNR

16. QC for environmental measurement
Why do we need quality control?
To prevent errors from happening
To identify and correct errors that have taken place
Why do we need quality control?
The purpose of QC is to identify and correct errors that may have happened in sampling, analysis, or data evaluation and to prevent additional errors from happening in the future. We need QCs to establish a level of confidence in our QA.
Discuss with the students some of the possible errors that could arise while sampling in the field…
while testing in the laboratory…
while analyzing data results in the office.
Briefly - how could some of these errors be discovered, corrected, and/or prevented?
EXAMPLES OF QUALITY CONTROL SYSTEMS PROVIDE:
Constant checks on sensitivity and accuracy of instrumentation and analytic systems
Interactive response - quality control information is used to maintain calibration and instrument response.
Real-time monitoring of instrument performance to ensure maintenance of calibrations and accuracy of results.
Control Charts - used to monitor long-term performance of measurement and analytical systems and initiate corrective action when biases are detected
QC is used to PREVENT and CORRECT ERRORS

17. QC: Internal vs. external measures
Internal quality control
“controllable” by those responsible for undertaking the project or directly “involved in the program”
External quality control
a “set of measures” established for and conducted by those people and organizations “outside of the program”
A good Quality Assurance Project Plan (QAPP) will provide for the validation of data and information through both internal and external QC measures.
Internal QC – Internal controls refer to those aspect of an environmental investigation that are “controllable” by those responsible for undertaking the project or directly “involved in the program” - in a Quality Assurance Project Plan (QAPP) - internal controls would refer to the group or organization that developed the QAPP and the employees, volunteers, or participants designated specific tasks within the QAPP.
External QC - refers to a “set of measures” established for and conducted by those people and organizations “outside of the program” (example: an audit or replicate sample collection conducted by the state environmental regulatory agency, the regional EPA representative, or their designated auditing group).

18. Quality control (QC): Internal
Internal Quality Control:
Equipment calibration
Proper training and certification of participants
Proper sampling and containment techniques
Proper data documentation
Photos are from the following sources:
1.) – asbestos containment chamber
2.) - recording data in the field
QC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Some examples of internal quality control measures implemented by the group responsible for the project (including but not limited to the collection and evaluation of samples and data) would include:
proper equipment calibration,
training and certification of field technicians and other staff,
proper documentation of all applicable information (see photo 2 – field logging of collected data),
prevention of cross contamination when collecting samples and data (see photo 1 – asbestos chamber)
Can your group list examples of other internal measures of “error” control?

19. Quality control (QC): External
External quality control:
Performance audits
Split sample analysis
Replicate (duplicate) sample analysis
Photos come from: 1.) - auditing computer work on screen
2.) person audit team
QC information has been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
External QC - “set of measures” established for those “outside of the program”
Examples of external QC controls include:
performance audits conducted by outside personnel, usually agency representatives or hired consultants
split sample analysis - where one sample is divided equally and sent to more than one laboratory (or annalist) for analysis, and
collection of “replicate” samples (or “duplicate” samples when collecting only 2 samples) - two or more samples taken from the same location, at the same time, using the same method, but independently analyzed; generally by a person(s) from outside the organization conducting the investigation.

20. Quality Assurance Project Plans (QAPP)
A quality assurance project plan (QAPP) is a project-specific QA document.
The QAPP outlines the QC measures used in implementing the project.
The QAPP is an invaluable planning and operating tool that should be developed in the early stages of the investigation, monitoring, or remediation project. Quality Assurance Project Plan (QAPPs) are used by agencies such as the EPA to ensure that the samples collected and analyzed, the data stored and manage, and reports that are written, are of high enough quality to meet project needs.
When agencies are involved in data collection a QAPP is always used to oversee the agency selection of laboratories, data collection methods, selection of parameters to be measured, consistency of data analysis and confidence in data quality. 
Few agencies will use data unless methods of data collection, storage, and analysis can be documented. Clear and concise documentation of procedures also allows a project to continue over time and space using the same methods. This is particularly important to a project that extends over a long period of time and that intends to establish a baseline of quality information that can be compared over time.
QAPP agency uses have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA

21. Quality Assurance Project Plans (QAPP)
The QAPP governs work conducted in the field, laboratory, and the office.
The QAPP:
guides the selection of project parameters and procedures
guides data management and analysis
provides steps to determine the validity of specific sampling or analysis procedures
designates both internal and external QC measures
The QAPP governs work in all aspects of the field, laboratory and office. The QAPP guides all areas of an investigation (field, lab and office) and includes:
the selection of parameters and procedures to be used in the investigation, including standards and methods to be used in both the field an laboratory (examples reviewed in the previous slides),
data management and analysis - how will those viewing the data assess the results, apply statistical evaluation, and make conclusions - which analytical methods are best for the data that needs to be obtained?
and, what steps are taken to determine the validity of specific sampling or analysis procedures – both quality assessment and quality control (QA/QC) – what conclusions are determined from the information?
Just as in the collection and evaluation of data, several agencies, programs and resources exist to describe how to develop and use a QAPP that is applicable to your area of study. We will talk about this in more detail in the following lesson when we discuss how to develop a QAPP and what key elements to include. However, for now we will concentrate on the WHY of QAPP development by discussing data objectives and key concepts necessary to implementing QA/QC – The PARCC Parameters (following slides).

22. QA/QC: Data objective and key concepts
Successful data collection and analysis is dependant upon “The PARCC Parameters”:
Precision
Accuracy
Representativeness
Completeness
Comparability
“The PARCC Parameters” include:
Precision
Accuracy
Representativeness
Completeness
Comparability
The PARCC parameters are the WHY of the QAPP and QA/QC. The QAPP is developed in order that the key concepts of a well run and thorough environmental investigation may be met.
The key concepts of QA/QC are the
“PARCC” Parameters – the WHY of the QAPP

23. Key concepts of QA/QC: Precision -
degree of agreement there is between repeated measurements of the same characteristic
can be biased – meaning there is a consistent error in the results
Accuracy -
measures how close data results are to a true or expected value – does not allow for bias
UMD med school
Photo taken from UMD med school – lab worker
Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Precision is the degree of agreement among repeated measurements of the same characteristic on the same sample or on separate samples collected as close as possible in time and place. Precision gives an idea of how constant and reproducible field or laboratory data are. However, this is only an indication of how consistent results are under similar conditions, and does NOT necessarily mean that results are a “true” value or accurate. (See target diagram on the following slide)
Precision may have a BIAS which is the degree of systematic error present in a process. Meaning, that when bias is present the result may be close together (precise) but will differ from the true value or expected result. (See target diagram on the following slide)
Accuracy is a measure of confidence in measurement. The smaller the difference between the measurement of a parameter and it’s true or expected value (the less bias), the more accurate the measurement and the more reproducible its result. (See target diagram on the following slide)

24. Key concepts of QA/QC: Accuracy
accuracy = (average value) – (true value)
precision represents repeatability
bias represents amount of error
low bias and high precision = statistical accuracy
Information adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Accuracy is the level of confidence in measurement. The smaller the difference between the measurement of a parameter and it’s true or expected value (the more accurate the measurement) the more reproducible its result. Accuracy is determined by comparing the result of analysis for a sample to the “known” value for that sample – such as a standard or a spike. Accuracy needs to be considered when choosing a method of analysis – What level of accuracy is needed? Grater accuracy allows for greater Repeatability - provides for a reliable measurement when you need to confirm the consistency of an environmental condition or look at change over time. ( e.g., for reliable interpretation of trends over time – the nitrification of a lake or remediation of a pollutant). Repeatability is usually acquired by specifying and sticking to a particular sampling and analytical method (methods discussed in slides 12 and 14).
Accuracy is reflective of both bias and precision (see target diagram). Discuss with the students the need for accuracy to be represented by both high precision and low bias.

25. Key concept QA/QC: Representativeness
extent to which measurements actually represent the true environmental condition or population at the time a sample was collected.
Representative data should result in repeatable data
Photos:
1.) taken from WOW Model 8 (8-4-03)
2.) taken from - Minnesota Lake
Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Representativeness – It stands to reason that the more accurate the data, the more representative it will be of the “true” or actual conditions at the site or area under investigation. As stated above, the more accurate the data, the more repeatable the results. Representativeness refers more to the sample location and selection. Are you really representing the conditions of the site when you choose specific sample or the specific sample location? (Simple Example: sampling for benthic organisms at the surface of a lake).
 Does this represent this?? 

26. Key concepts of QA/QC: Completeness
comparison between the amount of data intended to be collected vs. actual amount of valid (usable) data collected.
In the design of the QAPP – will the goals of the plan meet assessment needs?
Will sufficient data be collected?
Photo provided by NRRI
Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Completeness- is a data quality indicator generally expressed as a percentage. Completeness includes both the following (explanation and example for each provided below):
1.) an assessment of the amount of valid data obtained vs. amount planed for
2.) the design of a data collection plan to accurately assess the conditions
Deteriorated data gathering equipment will not gather usable data. – (see photo) Deterioration of equipment need not be this extreme. It can include such simple problems as the biofouling of in-place data loggers or the sticking of the wiper on a “self-wiping turbidity sensor. Or using equipment known to error under rigorous weather conditions. .
The “Valid Data” concept of completeness is important when you think of it as the amount of valid data obtained from a measurement system relative to the amount of valid data obtained under correct, normal conditions. In monitoring studies, it is not unusual to have some data that are invalid for a wide range of reasons (e.g., instruments that loose calibration, transcription errors, calculation/data reduction errors, lost samples, etc.). Generally, investigators try to minimize the amount of data lost because too much lost data can impact the statistical confidence of the analysis and cost extra money when data points need to be replaced, or whole datasets need to be discarded and re-sampled.
The “Sample Design” concept of completeness involves ensuring that the sample is designed to accurately assess conditions or trends across the entire population of interest. (e.g., using tools such as random sampling design or stratified sampling designs allow the investigator to fully characterize populations of interest; e.g., to fully characterize a fishery population you would want to sample at the lake AND its inflow and outflow streams, and you would want to look at all water layers when a lake is stratified - to avoid missing significant components in the population).
Would this give usable data ?? 

27. Key concepts of QA/QC: Comparability
the extent to which data can be compared between sample locations or periods of time within a project, or between projects
 Will similar data from these sites be Comparable ?? 
Photos:
1.) taken from
2.) taken from - Winter data
Key concept definitions have been adopted from U.S. Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA
Comparability - a method that works as well in fresh water may not work as well in saltwater (marine) environments! Different methods of analysis may have different limitations on their use because of the relative effects of different sample matrices. Another example - samples taken at different times of the day or at different seasons may not be comparable because of changes that occur with changing daylight or temperature.
By specifying a consistent method you can compare two of more places or two places in time - why is this important?( Hint: e.g., to allow reliable comparisons in “before and after” measurements, or “upstream and downstream” measurements around an effluent at a point source)
Discuss with the students whether on not data collected under these varying climate conditions may or may not be comparable? What sort of sampling events may be affected by these varying conditions.
Example: The collection of water samples for volatile organic compounds – how would hot weather conditions effect the samples and their volatile nature. How about in turn cold weather? What if the samples froze? What if collected samples froze and broke the bottles in which they were stored? Discuss how this relates back to the concept of completeness. Discuss how it relates back to the need for QCs to prevent these problems from occurring.

28. Review: QA vs. QC Quality Assurance (QA) broad program plan
establishes the need for QC
Quality Controls (QC)
standardized tests and methods
the “HOW” of QA
Use this slide as a opportunity to briefly review the meaning of QA and QC – What are the key differences and similarities between the two?
Quality Assurance - a broad plan for maintaining quality in all aspects of a program, also refers to the overall management system and includes: Organization, Planning, Data collection, Documentation, Evaluation, and reporting activities. All designed to address the success of the environmental assessment.
Quality Control refers to the routine technical activities and established check and balances, the essential purpose of which is to control error. QC employs standardized tests and methods used in the field, laboratory, and office to generate quality, consistent, and repeatable data.

29. Review: Quality Assurance Project Plans
The QAPP is a project-specific QA document.
The QAPP outlines the QC measures to be taken for the project.
QAPP guides:
the selection of parameters and procedures
data management and analysis
steps taken to determine the validity of specific sampling or analysis procedures
Review with students the need for a QAPP (introduced in slides 15 to 21)

30. Elements of a QAPP
The QAPP governs work conducted in the field, laboratory, and the office.
QAPP can be broken out into four areas of the project
Project management (office)
Measurement and data acquisition (field and lab)
Assessment and oversight (field, lab, and office)
Data validation and usability (field, lab, and office)
Just as we discussed in the beginning of this presentation, QA/QC enters into all aspects of a project design, assessment, and evaluation. The QAPP governs actions to be taken in all aspects of the project including the field, laboratory, and the office. To assure a high quality investigation, consistent means of data acquisition and evaluation must be followed in all areas of the investigation.
A following lecture (13C) will concentrate on the development of a QAPP. How is one developed, by whom, and what to include and what considerations need to be addressed when compiling such a document and outlining such a program.

31. Review: QA/QC key concepts
“The PARCC Parameters” are the WHY of QA/QC and assure successful data analysis:
Precision
Accuracy
Representativeness
Completeness
Comparability
The QAPP is most specific on details surrounding data acquisition and evaluation, or work conducted in the field and laboratory. To assure that data gathered and analysis conducted is of high quality and consistency, the PARCC are employed.
Review again briefly with the students the “PARCC Parameters” and what sort of QC measures can be used to assure the obtainment of each including:
Precision
Accuracy
Representative (Repeatability)
Completeness
Comparability
How can one be aware of the QC measures that are to be used during implementation of the project? How can one make sure they are using the QC methods properly? BY REVIEWING AND REFFERING BACK TO THE QAPP.
The following lecture (13B) will go into more detail on the “Whys” of implementing the QC portion of a project plan and some of the “How” behind the whys. We will introduce Data Quality Objectives (DQOs) and discuss how we evaluate quality of data. We will discuss QC sample(s) applicable to assess for each key parameter and touch upon statistical calculation of percussion along with determination of accuracy and bias
The key concepts and WHY of QA/QC are the
“PARCC” Parameters

32. References
EPA 1996, Environmental Protection Agency Volunteer Monitor’s Guide to: Quality Assurance Project Plans EPA 841-B , Sep 1996, U.S. EPA, Office of Wetlands, Washington, D.C , USA

33.
This is the closing WOW slide that should appear at the end of all WOW presentations.
This slide should not be altered in anyway. Do not add any additional elements to the slide, alter the web address, add additional copy, etc. 18 4 4