1. ...Green Technology through Innovation Ensuring Data Accuracy at the Lowest Limit Possible Presented by: Judy Morgan, MS, REM VP, Chief Regulatory Officer ESC Lab Sciences Mt. Juliet, TN jmorgan@esclabsciences.com

2. ...Green Technology through Innovation Accuracy: How Low Can YOU Go? Accuracy* - ˈa-kyə-rə-sē freedom from mistake or error : the quality or state of being accurate the ability to work or perform without making mistakes *http://www.merriam-webster.com/dictionary/accuracy TCEQ Environmental Trade Fair May 3-4, 2015

3. ...Green Technology through Innovation Accuracy – Lab QC Accuracy is how well a measured value agrees with the expected or true value Typically expressed as % Recovery As true value is rarely known for environmental samples, accuracy is determined using environmental samples or reagent water spiked with a known amount of the target analytes TCEQ Environmental Trade Fair May 3-4, 2015

4. ...Green Technology through Innovation Accuracy – Instrument Detection Sensitivity is the ability of an instrument or procedure to accurately detect an analyte at a specific concentration Method Detection Limit or Limit of Detection (MDL or LOD) The minimum measured concentration of an analyte that can be reported with 99% confidence that the result is greater than zero The lowest concentration that can be determined to be statistically different from a blank. MDL procedure assumes that blanks are centered around zero. Where blanks are not centered around zero, the MDL will be too low and false positives will result The current MDL procedure does not provide assessment for Type I (false positives) or Type II (false negatives) errors and gives no consideration to blank performance or instrument noise/background. TCEQ Environmental Trade Fair May 3-4, 2015

5. ...Green Technology through Innovation Accuracy – Instrument Detection Limit(s) of Detection (LOD): The smallest concentration of a substance that must be present in a sample in order to be detected at the DL with 99% confidence. At the LOD, the false negative rate (Type II error) is 1%. An LOD may be used as the lowest concentration for reliably reporting a non-detect of a specific analyte in a specific matrix with a specific method at 99% confidence. Limit(s) of Quantitation (LOQ): The smallest concentration that produces a quantitative result with known and recorded precision and bias. The LOQ should be 1 to 2 times the concentration of the lowest initial calibration standard and within the calibration range. The concentration of an analyte that can be accurately measured with 99% confidence in the value of the measurement. Should be at least verified through the analysis of a standard reference material. TCEQ Environmental Trade Fair May 3-4, 2015

6. ...Green Technology through Innovation Precision Precision - How consistently one can repeat the same action……even if it’s wrong! TCEQ Environmental Trade Fair May 3-4, 2015

7. ...Green Technology through Innovation Precision – Lab QC Precision is how closely multiple measurements of the same material under the same conditions agree with each other Generally expressed as Relative Percent Difference (RPD) Determined from duplicate spiked analyses (MS/MSD or LCS/LCSD) or from analyzing homogenous samples in duplicate where the target analyst is present in the sample matrix TCEQ Environmental Trade Fair May 3-4, 2015

8. ...Green Technology through Innovation Ideal – Good Precision and Accuracy The same correct result…..over and over. Good for: Target Practice Basketball Dart games (winning ) Lab Results

9. ...Green Technology through Innovation All instruments can achieve the same detection and sensitivity All laboratories perform the analyses exactly the same, because the approved methods are the same Regulatory limits drive the ability of technology to meet requirements TCEQ Environmental Trade Fair May 3-4, 2015

10. ...Green Technology through Innovation Analytical Detection Limitations – “A Reality Check” All instrumentation have limits that can vary widely Different types of instruments present different analytical challenges Simple instruments produce a response vs concentration Complex instruments produce data using 3 or more analyte specific responses Different types of instruments present different analytical challenges Simple instruments produce a response vs concentration Complex instruments produce data using 3 or more analyte specific responses All laboratories have different environmental & physical conditions that can affect: Instrument background Signal to Noise levels Potential for false positives and negatives All laboratories have different environmental & physical conditions that can affect: Instrument background Signal to Noise levels Potential for false positives and negatives More on this later…… TCEQ Environmental Trade Fair May 3-4, 2015

11. ...Green Technology through Innovation Laboratory Physical Environment – “The Affect” Monitor and provide controls for: Humidity, Temperature, Barometric Pressure, Power surge, Cross Contamination, Good Housekeeping practices……. Suggested Controls: Dedicated Air Handling Zone Temperature Control Solvent recovery and vapor control Separate spaces/buildings Power generators and/or Uninterupted Power Supply Systems TCEQ Environmental Trade Fair May 3-4, 2015

12. ...Green Technology through Innovation Environmental Impact on Instruments & Blanks Blank Threshold Defined: The value that represents the highest response expected when no analyte is actually present.  Represents 99% confidence that no false positives will be reported above the MDL or LOD.  Blank threshold represents instrument fluctuation in normal daily responses when no analyte is present  Determined by analyzing blanks, calculating the mean response & standard deviation and using the following equation: Instrument stability is affected by environmental factors such as: room temperature, humidity, power fluctuations, cleanliness, air particles, etc. Instrument response is affected by improper maintenance schedules or practices Blank Threshold = mean of blanks + (Students t x Standard Deviation of the Blanks) TCEQ Environmental Trade Fair May 3-4, 2015

13. ...Green Technology through Innovation Data User Assurance The goal of analysis is to provide: Results with good precision and accuracy Data of known and documented quality for legal defensibility The goal of project specifications or Data Quality Objects (DQO) is to ensure: Reporting and detection limits are met Approved methods are used Necessary accreditations are in place and current Project Specific Quality requirements are met Timelines, service expectations, and other contract specifications are met. TCEQ Environmental Trade Fair May 3-4, 2015

14. ...Green Technology through Innovation LOQ shall be set at or above the concentration of the lowest initial calibration standard and within the calibration range Set LOQ and verify within method criteria at least quarterly LOD = lowest concentration for reliably reporting a non-detect of a specific analyte in a specific matrix with a specific method at 99% confidence Data User – Verifying Project Required Reporting and Detection Limits Laboratory Method Validation (Init. Calibration, Accuracy, Prec., IDOCs, MDLs, etc.) Blank Analysis Calculate Blank Threshold: Mean Blank Response + (Students t X Standard Deviation of Blanks) Is the MDL above the Calculated Blank Threshold? Compare Blank Threshold value to Laboratory MDL Use MDL as calculated and perform LOD Verification Set LOD at or above the Blank Threshold and verify. Compare Project Required Reporting and Detection Limits Yes No TCEQ Environmental Trade Fair May 3-4, 2015

15. ...Green Technology through Innovation Blanks Good: MDL/LOD  LOD Ver.  LOQ MDL/LOD LOD Verification LOQ Blanks: Average = 0.566 Std dev = 0.530 3SD = 1.67 Verified at 3 to 4 X the MDL TCEQ Environmental Trade Fair May 3-4, 2015

16. ...Green Technology through Innovation Blanks Bad Reality: MDL/LOD  LOD Ver.  LOQ MDL/LOD LOD Verification LOQ Blanks: Average = 1.32 Std dev = 1.45 3SD = 4.57 TCEQ Environmental Trade Fair May 3-4, 2015

17. ...Green Technology through Innovation Issues by Instrument Type: Simple vs Complex Simple Instruments (Typically inorganic analysis): Spectrophotometers and some spectrometers Measurements are based simply on analyte concentration as compared to an instrument response. UV/Vis, AA, ICP, ICP/MS, CVAA, TOC, etc. Ion Chromatography (even though retention time, peak height, and width are generated) Note: The analyte is indistinguishable from the background until a certain threshold is met. Highly prone to false positive responses. TCEQ Environmental Trade Fair May 3-4, 2015

18. ...Green Technology through Innovation Issues by Instrument Type: Simple vs Complex Complex Instrumentation (Typically organic analysis): GC, GC/MS and related: Instrumentation that perform analyte identification using multiple sources of identification criteria, such as: Peak width and height Peak shape & pattern recognition Retention time Ions or ion ratios Not as prone to false positives as simple instrumentation. TCEQ Environmental Trade Fair May 3-4, 2015

19. ...Green Technology through Innovation Data User – So, what does all of this mean? Analyte detection is not Black or White Differences can exist, from: Lab to lab Instrument to instrument Prep to prep Background must be considered and evaluated for each individual instrument TCEQ Environmental Trade Fair May 3-4, 2015

20. ...Green Technology through Innovation Metals Analysis by ICP A Comparison between Instrument Manufacturers Vendor 1 Arsenic Analysis Reporting Limit – 20ppb Calculated MDL – 4.4 ppb Blank Average – 3.6ppb Blank SD – 2.9 Blank Threshold – 10.2ppb Blank Threshold > MDL Reporting limit is only 2x the Blank Threshold Vendor 2 Arsenic Analysis Reporting Limit – 20ppb Calculated MDL – 2.2 ppb Blank Average – 0.93ppb Blank SD – 0.495 Blank Threshold – 2.10ppb Blank Threshold < MDL Report data at 20ppb Evaluate data to 2.2ppb TCEQ Environmental Trade Fair May 3-4, 2015

21. ...Green Technology through Innovation Metals Analysis by ICP A Comparison between Instrument Manufacturers Vendor 1 Lead Analysis Reporting Limit – 5ppb Calculated MDL – 2.0 ppb Blank Average – 2.0ppb Blank SD – 1.3ppb Blank Threshold – 4.99ppb Blank Threshold > MDL Reporting Limit is essentially equal to the Blank Threshold (making false positives likely) Vendor 2 Lead Analysis Reporting Limit – 5ppb Calculated MDL – 1.8 ppb Blank Average – 0.459ppb Blank SD – 0.342ppb Blank Threshold – 1.27ppb Blank Threshold < MDL Report data at 5ppb Evaluate data to 1.8ppb TCEQ Environmental Trade Fair May 3-4, 2015

22. ...Green Technology through Innovation ICPMS Arsenic Analysis Reporting Limit – 1ppb Calculated MDL – 0.051ppb Blank Average – 0.0300ppb Blank SD – 0.055ppb Blank Threshold – 0.157ppb Blank Threshold > MDL LOD must be set above Blank Example: LOD = 0.2ppb LOD Verification = 0.5ppb Report data to 1ppb Evaluate data to 0.2ppb TCEQ Environmental Trade Fair May 3-4, 2015

23. ...Green Technology through Innovation UV/Vis and other Spectrophotometers Blanks can exhibit consistent absorbance at the “zero” concentration Simpler than ICP: background fluctuation is less Must determine the concentration where the absorbance is due to analyte presence TCEQ Environmental Trade Fair May 3-4, 2015

24. ...Green Technology through Innovation Complex Instrumentation Organics – GC, GC/MS Typically do not have the same false positive issues as simple instrumentation Low to no background, typical response is “Non-Detect” (ND) Instrument threshold setting for ND is based on area count Must meet method and instrument identification requirements (minimum number of ions and associated ratios, retention time, resolution, tuning, etc.) As concentration decreases, ID indicators decrease proportionally, thus detection decreases and eventually disappears. TCEQ Environmental Trade Fair May 3-4, 2015

25. ...Green Technology through Innovation Data User: 1.My lab is accredited 2.My lab sent me a list of their MDL’s and Reporting Limits 3.My QC passes acceptance limits 4.My lab used the correct method and have run it for years 5.My lab rarely needs to qualify my data 6.My lab performs well on proficiency tests How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

26. ...Green Technology through Innovation Data User: 1.My lab is accredited Accreditation is designed to ensure that a laboratory has the systems in place to produce data of known and documented quality. Assessment provides for a sampling of the laboratory capability, but in no way addresses 100% of the data being produced There is currently no definitive requirement in 40 CFR Part 136, Appendix B to review or assess blank performance relative to MDL or LOD determination. How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

27. ...Green Technology through Innovation 2.My lab sent me a list of their MDL’s and Reporting Limits A list of MDL’s and Reporting Limits does not show performance or calculation As demonstrated, without thorough review of the data, including blank evaluation, a list will not reveal true instrument performance Blanks are the starting point to establish true detection with confidence. Data User: How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

28. ...Green Technology through Innovation 3.My QC passes acceptance limits Typical quality control samples are spiked at the mid-range of the calibration and would not be as affected by background or false positives/negatives Data User: How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

29. ...Green Technology through Innovation 4.My lab used the correct method and have run it for years Methods do not directly require evaluation of instrument noise to establish reporting limits. Methods typically address the ability to detect environmental levels of analytes. Methods do not focus on the ability to accurately produce trace level analysis Methods do not focus on specific instrumentation and offer only general requirements for performance. Equipment sensitivity has surpassed the ability of most current approved methods and must be considered through more comprehensive validation practices with a focus on cleanliness and detection contributors. Data User: How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

30. ...Green Technology through Innovation 5.My lab rarely needs to qualify my data Qualifiers are designed to provide additional information regarding sample analysis; both good and bad. Lack of use diminishes the ability of the data user to make informed decisions. The reporting of qualifiers, by the laboratory, is on an “honor” system. There is no guarantee that labs consistently provide this information. Results reported between the MDL and Reporting limit must always be qualified. The lack of reporting qualifiers, by laboratories, is one of the TOP 10 deficiencies cited by the nations largest accreditors. Are you getting the whole picture? Data User: How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

31. ...Green Technology through Innovation 6.My lab performs well on proficiency tests Proficiency tests are designed to assess laboratory performance at the level of the intended environmental compliance program. The unknown levels are not typically at a concentration that is influenced by diminished instrument performance at the detection level. Data User: How Do I Know My Requirements are Being Met? TCEQ Environmental Trade Fair May 3-4, 2015

32. ...Green Technology through Innovation Importance of Accuracy at the Lowest Level Some programs require reporting to the MDL for all methods, all analytes, and all samples These results are used to make environmental decisions So: What can we expect from results above the RL or PQL? Between the MDL and RL/PQL? Below the MDL? TCEQ Environmental Trade Fair May 3-4, 2015

33. ...Green Technology through Innovation 2015 Methods Update Rule 40 CFR Part 136 Appendix B does not currently address a definitive requirement for blank assessment. The update to 40 CFR Part 136, proposed on Feb. 19, 2015 has a revised MDL procedure. The update includes a requirement for blank analysis and assessment. Requires the calculation of a Blank threshold, called Method Detection Limit Blank (MDL B ). Sets criteria for the relationship of the MDL B to the MDL and Reporting Limit. TCEQ Environmental Trade Fair May 3-4, 2015

34. ...Green Technology through Innovation Conclusion Data results are necessary for compliance at all levels Many times the results have to be reported to the MDL Precision and accuracy are necessary at all levels False positives and negatives must be considered in order to have defensible data at the lowest levels possible Blank analysis and Blank Threshold are key to showing true detection at the lowest limit MDLs are validated, based on comparison to the blank responses, provide more reliable and accurate analyte results Reporting limits (RL, PQL, LOQ) are established in relation to the verifiable MDL Current MDL procedure fails to assess blank effects on low concentrations Proposed MDL procedure addresses the issue clearly All MDL  RL data must be qualified Where a data user needs extremely low limits, request to see the laboratory data that shows the relationship of blank performance, to MDL, to reporting limit. They should represent building blocks. TCEQ Environmental Trade Fair May 3-4, 2015

35. ...Green Technology through Innovation THANK YOU! Any Questions? Contact: Judy Morgan, MS, REM VP, Chief Regulatory Officer jmorgan@esclabsciences.com