1. Untapped Power of Data Validation
Texas Commission on Environmental Quality
Environmental Trade Fair and Conference (ETFC)
Austin Convention Center
May 16-17, 2017
Erin E. Rodgers – Environmental Standards, Inc.

2. What, When, Why Data Validation? Data Validation Case Studies
Agenda
What, When, Why Data Validation?
Data Validation Case Studies
Forensic Data Review Case Study
Conclusions 2

3. Why Data Validation?
There are many common misconceptions regarding environmental laboratory data. For example, do you believe any of the following statements?
The laboratory is accredited; therefore, my data must be correct, accurate, and precise.
The laboratory follows the published analytical method; therefore, my data must be acceptable.
Only the laboratory impacts the quality of the analytical data. 3

4. Why Data Validation?
Many costly and important remedial decisions are made based on environmental analytical data.
The quality of the analytical data will directly impact the overall project decisions that are made on the basis of the data.
Project data drive remediation and remediation costs. 4

5. When Perform Data Validation?
Potential for litigation?
Potential for high level of scrutiny?
CERCLA or Superfund
Development / Redevelopment of Potentially Contaminated Site
Regulatory Driven
Issues Identified or Questions Arise During Limited Data Review 5

6. When Perform Data Validation?
Limited data validation is routinely performed
End data user desire to understand the usability or acceptability of the data
Can be automated
Full Data Validation is performed usually due to one of the reasons listed on the prior slide
Most of the issues that could cause substantial delays, liability, and cost impacts are identified during full validation
Always performed manually 6

7. What is Data Validation?
Several levels or stages of validation
Limited data validation
“Checklist” Validation
Reviewing QC to determine appropriate qualification and use of the data
Stage 2A - Summary form review of batch QC analyses
Stage 2B - Summary form review of batch QC analyses and calibrations/instrumental analyses
Full data validation
Comprehensive, critical review of data
Stage 4 – Includes everything in Stage 2 plus a critical review of raw data and instrumental outputs to qualitatively and quantitatively confirm the reported results 7

8. Typical Data Package Limited Data Package Full Data Package
Summary Forms Only
Sample Results
Batch QC Results
Method Blank
Blank Spike
Matrix Spike
Field/Laboratory Duplicate
Summary Forms & Raw Data
Sample Results
Batch QC Results
Method Blank
Blank Spike
Matrix Spike
Field/Laboratory Duplicate
Calibrations
Tunes, Initial, Continuing Checks, Run logs
Instrumental QC
Internal Standards, Interference Checks, Serial Dilutions, Dual-Column Precision
Extraction, Digestion, and/or Preparation logs 8

9. What is Data Validation?
Process of critically examining laboratory data
Includes review of:
Verification of Correctness
Compliance
Usability
In this presentation, we are going to discuss issues identified during Stage 4 or full validation review
Full validation is the process of reviewing data for correctness, compliance and usability
Lower levels or stages of validation typically only encompass the usability aspect of validation. Validation is often performed to “check the box” only encompasses a usability assessment wherein “flags” are appended to the analytical data.
In this presentation I will detail issues that were identified during the correctness or verification assessment and compliance review aspects of validation that saved the client time, money, and potentially additional parameters being added to the COC list for a particular site 9

10. What is Data Validation?
Correctness Assessment
All samples collected/requested were analyzed and reported
All analytes that were analyzed/requested were reported
Correct methods, MDLs/RLs, units, basis
Confirm qualitative identification of target compounds
Recalculate reported concentrations from raw data
AKA – “Verifying” the analytical data 10

11. What is Data Validation?
Compliance Assessment
Compliance with analytical method
Compliance with project control documents
Quality Assurance Project Plan (QAPP)
Sampling and Analysis Plan (SAP)
Work Plan
Project Specifications
Compliance with permit requirements
Compliance with regulatory requirements 11

12. What is Data Validation?
Usability Assessment
Review of QC, calibrations, holding times, and sample receipt
Determines appropriate “flagging” of data
Blank Qualification – U or B
Rejection of Data – R
Estimating Data – J
Information to data user on the potential limitations and appropriate use and potential limitations of the analytical data
All stages of validation include a usability review. Data may be qualified for a number of reasons during validation. This presentation will focus on the correctness and compliance assessments performed during Stage 4 or full validation, rather than detailing qualifications applied during a usability assessment. 12

13. DV Case Study #1 Transcription Errors – Most Common Errors
Impact Reported Results
Incorrect Dry-Weight Reported
Percent solids reported as 7%
Should have been reported as 75%
Results were reported on a dry-weight basis
Concentration
of metals in soil samples decreased
10-fold.
Samples were soil samples that all had % solids values in the 70 to 80% range. One result stuck out as having very low % solids at less than 10% solids. This did not seem realistic knowing the matrix of the samples. Review of the percent solids raw data showed that the % solids should have been reported as 75%
Analyte
Initial Results (7% solids)
Revised Results (75% solids)
Benzo(a)pyrene
0.054
0.0051
Antimony 14
1.32
Arsenic
4.62
0.437
Lead
840
79.3 13

14. Issue #1 Continued Transcription Errors - Incorrect Initial Weight
Manual transfer of initial sample weights to the laboratory information management system (LIMS) for final concentration
LIMS preparation log reported initial weight as g
Handwritten preparation log reported initial weight as g
Metals concentrations increased 1.5 fold
Transcription errors become a real issue when manual transfer of values occur throughout several areas of the laboratory. Some labs or some areas of a lab have direct transfer of balances to LIMS. Other labs or lab sections (wet chem lab) must manually record the sample weight and transfer this weight into the LIMS system manually. Transcription errors are often seen with manual transcription.
units = mg/kg
Reported concentration
Recalculated concentration
Reported MDL
Recalculated MDL
Reported RL
Recalculated RL
antimony
0.352
0.521
0.163
0.241
0.678
1.00
arsenic
0.477
0.705
0.339
0.502
lead
9.64
14.3
0.23
0.341
0.542
0.80 14

15. of metals in soil samples decreased.
Issue #1 Continued
Reporting Errors – Dry vs. Wet Weight Reporting
Samples were air-dried and sieved upon receipt
Should have been reported on an as-received basis (100% solids)
All results were reported on dry-weight basis, using the percent solids from the received sample (prior to
air-drying and sieving)
Not included on this slide but another example of a reporting error that I’ve run across was where all metals results in all soil samples were corrected for percent moisture, rather than percent solids. The resulting data were revised and re-reported and the concentration of metals in the samples decreased 2 to 4-fold.
Concentration
of metals in soil samples decreased.
Analyte
Original Result
Revised Result
Dry-Weight Corrected
Air-dried
Aluminum
3560 mg/kg
2980 mg/kg
Antimony
17.8 mg/kg
15 mg/kg
Arsenic
63 mg/kg
52.8 mg/kg
Barium
18.6 mg/kg
15.6 mg/kg 15

16. Resolution
Laboratory was requested to review the observed discrepancies
Data were revised and re-reported with the corrected concentrations
One instance resulted in significantly lower reported concentrations
One instance resulted in slightly lower reported concentrations
One instance resulted in slightly higher reported concentrations
Issues observed during quantitative review – recalculation of results from raw data 16

17. DV Case Study #2 Misidentified PCB Aroclor
Qualitative identification issue
Review of PCB pattern and ratio of
peaks in sample versus PCB standard
Overlapping/similar peaks present in both Aroclors
Submitted a blind performance evaluation (PE) sample to the laboratory with environmental PCB samples to verify PCB reporting
Identified by qualitative review of the chromatogram 17

18. Resolution
Critical review of sample raw data/chromatograms and PE sample qualitatively identified and confirmed the correct Aroclor pattern present in the samples
Laboratory revised and reissued data with correct Aroclor reported prior to data submission
Saved client by reporting the correct detected Aroclor
Reporting incorrect Aroclor detection may have added an additional COC to the parameter list as the incorrect Aroclor was not known to be used at the site historically 18

19. DV Case Study #3 False positive result reported – Improper Integration
Incorrect integration of the analyte by the instrument
Identified during qualitative review of chromatogram
Before After 19

20. Resolution
Laboratory was requested to review integration and revise data.
Laboratory originally reported positive result.
Revised result was ND < RL.
Saved the client by not reporting a false positive result which may have expanded the COC list to include the analyte. 20

21. DV Case Study #4 False positives – Method Limitation
Positive results for hexavalent chromium were reported for samples, but the matrix spike recoveries were 0%, indicating a reducing sample matrix
For ultraviolet-visible spectrophotometry (UV/VIS), dark/turbid extracts often produce a signal that may be reported as false positive results
Technical expertise – 0% MS recoveries coupled with dark extracts
7196A UV/VIS method
Issue was noted when the sample digestions included notes stating that the extracts were very dark in color and turbid
Knowing that dark extracts can lead to color interferences the data reviewer noted that the MS/MSD recoveries were 0% for the soil sample. The LCS recoveries were spot-on. 0% MS/MSD recoveries for hex chrom often tells you that the samples are reducing in nature and hexavalent chromium cannot exsist (is quickly reduced to trivalent chromium). 21

22. Resolution Samples reanalyzed by a different method
Ion chromatography physically removes extract interferences
Ion chromatography confirmed that hexavalent chromium was not present in the samples
7199 IC method
Physically removes extract interferences by absorbing organics and suspended solids through the use of a guard column
Anion exchange column separates the remaining constituents in the extracts
This analytical method and separation technology yields better qualitative identifications, with less need for extract dilution, resulting in more accurate and precise results
In this specific example, a study was also performed to determine whether the soil samples were reducing in nature as hexavalent chromium cannot exist in nature under reducing environmental conditions. In order to determine whether the samples were reducing, pH an ORP were performed.
Another line of evidence used in this example, was to request and analyze total chromium which wasn’t previously being done. We were able to show that the total chromium results were less than the hexavalent chromium results providing another line of evidence that the color interferences were leading to false positive results. 22

23. DV Case Study #5 Method Non-compliance Led to Potential NOV
Identified during compliance evaluation
Single BOD sample way outside normal results
Logistics and timing issue prevented additional samples from being collected
Agency started process for permit violation
Review of raw data noted BOD dilution water blank reading was 2.2 mg/L
Method requires BOD < 0.2 mg/L in dilution water blanks
Elevated blank reading resulted in inaccurate and unreliable quantitation of BOD results
Method 5210B Section 6 QC Checks C – Dilution Water quality check. Serves as a check on quality of unseeded dilution water and cleanliness of incubation bottles. The DO update in 5 d MUST not be more than 0.2 mg/L before making seed corrections. If the dilution blank exceeds 0.2 mg/L, discard all data for tests using this dilution water. 23

24. Resolution
Laboratory requested to review and verified the analytical error.
Laboratory formally rescinded analytical results.
Laboratory contacted the reporting agencies on client’s behalf.
NOV was rescinded.
Data were non-compliant with analytical method. 24

25. DV Case Study #6 Insufficient field sample preservation.
Identified during compliance evaluation
TOC method requires samples to be preserved to pH < 2.
Received with pH values ranging from 4 – 6.
Sample preservation was method non-compliant.
Method does not have an allowance for laboratory preservation like metals analysis. 25

26. Resolution Current data set was qualified during validation.
Quick study was performed in the field to determine sufficient amount of preservative needed to properly preserve the samples to pH < 2.
Training provided to field staff so future samples were properly preserved and method compliant. 26

27. DV Case Study #7 Multiple Major Method Non-Compliances – Metals Data
Improper calibration sequences
Failure to analyze initial calibration blank (ICB)
Failure to analyze continuing calibration verification (CCV) and continuing calibration blank (CCB) at method-specified frequency
Failure to digest and analyze QC samples at method-specified frequency
Failing QC analyses
Failure to analyze several method-required QC analyses 27

28. Resolution
Laboratory Case Narratives indicated that all metals results should be rejected due to the many compounding issues.
This assessment was confirmed by validators when reviewing the packages.
Retained metals samples were pulled from the laboratory and sent to another laboratory for analysis.
Saved by the long holding time for metals! 28

29. DV Case Study #8 Blank contamination for same analytes
Identified during usability assessment
Laboratory water contained several analytes above MDL.
Laboratory supplied deionized water for field & equipment blanks.
Same water used for laboratory method blanks.
Consistent levels observed for several analytes in the various blanks.
Similar levels in blanks and samples would have led to many results qualified for blank contamination. 29

30. Resolution
Laboratory requested to investigate the consistent low-level positive results in blanks.
Analysis of the laboratory DI water indicated background levels of several constituents.
Water source run through an additional polishing process to remove background contaminants.
Constituents observed in DI water were not used to qualify sample results.
Active data management enabled the data validation chemists to quickly identify the issue and observe the consistent level of constituents reported in the various blanks. 30

31. Conclusions
Common data issues are not often identified during a cursory review.
Oftentimes validation is performed to “check the box.”
Lower levels of validation are typically performed to determine the usability of the result.
Full, comprehensive validation includes review for correctness and compliance.
Most issues were identified during the correctness/verification or compliance review. 31

32. Forensics Case Study
Natural gas condensate released onto residential properties
Review of SVOC analysis full scan and PAH data to determine when cleanup activities successfully removed source material
Evaluation of remaining PAH concentrations to determine petrogenic versus pyrogenic source 32

33. Forensics Case Study SVOC chromatogram – impacted soil sample
Large upward hump characteristic of source material 33

34. Forensics Case Study
SVOC chromatogram – same location after excavation and removal of impacted soil
Large upward hump not present – source material removed 34

35. Forensics Case Study
SVOC analysis indicated source material was successfully remediated.
PAH constituents remained – present in confirmation/after excavation samples.
PAHs were present in most samples, but without a discernable pattern.
i.e., PAHs were not necessarily higher closer to the source of the release; some of the highest PAH concentration was far away, outside of the reasonable mist deposition area. 35

36. Forensics Case Study
Selected a class of high molecular weight compounds to review.
These compounds are known to be products of incomplete combustion.
If we could show that these compounds were not present in the source material, but present in the samples, then we could show that the presence of these compounds was not due to the released material and were, instead, “background” or present due to other environmental factors. 36

37. Forensics Case Study
Sample collected outside of the mist deposition area. Some of the highest PAH results reported.
“Background” Soil
Pattern of high mw doublets and singlets are present here, outside the mist deposition (below). Characteristic of background prior to release 37

38. Forensics Case Study Source oil sample
High molecular weight compounds not observed
Pattern of high mw doublets and singlets not present in source material 38

39. Forensics Case Study Visually impacted soil sample
Same sample shown under SVOC analysis
Pattern of high mw doublets and singlets not present 39

40. Forensics Case Study Same sample after excavation/cleanup
Source confirmed removed by SVOC analysis
Pattern of high mw doublets and singlets are present here. 40

41. Forensics Case Study - Conclusions
SVOC full scan chromatograms showed that the excavation of the impacted soil was successful in removing the source material.
PAH SIM chromatograms developed by forensics team showed that petrogenic PAHs of concern were not present in the source material.
PAH SIM analysis showed that petrogenic PAHs were ubiquitously distributed and not associated with source material. 41

42. “Setting the Standards for Innovative Environmental Solutions”
Thank You
“Setting the Standards for Innovative Environmental Solutions”
Environmental Standards, Inc.
1140 Valley Forge Road