1. 1 Guest Speaker: Bill Frietsche US EPA

2.  April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp  April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) – Travis Maki  April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe  April 28: Calculating Bias and Precision and AQS reports – Bill Frietsche  May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring  May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell  May 19: 40 CFR 58 App. A- PM filter and continuous methods –Brandy Toft 2

3. …the difference between your answer and the “truth” Two components of total error (accuracy):  Bias (jump)  Precision (wiggle) 3

4.  Some imprecision is unavoidable  Sometimes up, sometimes down– “random”  Difference divided by best estimate of the truth  “truth” is: ◦ For gas QC checks: known conc. ◦ For PM flow rate: audit (known) FR ◦ For PM2.5 collocated: their average 4

5. 5 d i = the difference between a known value and your value (flow rate, conc., voltage, that should remain the same) Sequence of check (date, time, check number) Repeated measurement of the same thing

6.  Pairs of simultaneous measurements (collocated instruments) 6 d i (difference between collocated values) date

7. 7 Can be sudden or due to slow drift over time: d i = the difference between the true value (audit value) and the analyzer date

8.  Includes both bias and precision  At any one time, an audit value could be close to the analyzer value due to precision errors in both  Or, audit value could be far from analyzer value due to precision errors adding  Audits estimate Accuracy  EPA uses audit results to verify that ongoing QC checks do represent total error 8

9.  Auditor’s see only one day, while your QC checks see overall big picture 9

10.  1-pt QC checks no longer called precision checks, because the results are used (by YOU) to calculate both precision and to estimate total error  Each check <= 7% is the CRITICAL criteria for each set of data since last passing check  The audits basically verify your precision and bias that have been calculated all along from your QC checks (concentration and flow rate, for PM) 10

11. MethodPollutantsFrequencyMQO One-Point QCSO 2, NO 2, O 3, CO Every 2 Weeks O 3 : Precision = 7% SO 2, NO 2, CO : Precision = 10% Flow Rate Verification (QC check) PM 10 (lo- Vol) PM 2.5 Once every 4 weeks <= 4% of Standard Flow Rate Verification (QC check) PM 10 (high- Vol), TSP Once per quarter <= 7% of Standard Collocated Sampling PM 10, TSP, PM 2.5 15% of Network Every 12 Days Precision CV= 10% (which means RPD =14%) 11 *and total error, more on this later….

12.  Estimates your overall system uncertainty, if you were perfectly calibrated  Changes over time can point to operator error, lab error, poorly written procedures, equipment/standards going bad  Changes can be fixed sooner rather than at audit  EPA compares precision by site and agency 12

13.  Useful to track precision (or bias) over time  See this example (and you fill in your values) at:  Another example in the DASC tool: 13 didi

14.  The DASC tool automatically writes your values from the columns into the chart: ◦ Measured value is your analyzer ◦ Audit value is the known 14 You may want to add dates to the x-axis, or use my example if you want to set up control charts in your program didi

15.  In their annual QA reports, from your RP transactions in AQS:  http://www.epa.gov/ttnamti1/qareport.html http://www.epa.gov/ttnamti1/anlqa.html http://www.epa.gov/ttnamti1/qareport.html http://www.epa.gov/ttnamti1/anlqa.html 15 didi

16.  Recommendations in redbook say:  Critical criteria invalidate every hour that is not met  Operational criteria means something probably wrong, go check it  Systematic criteria mean as a set (day? year?) data is not usable for NAAQS, but individual hours or more may be valid 16

17.  (NO2 and SO2 are the same) 17

18. 18 MethodPollutantsFrequenc y MQO Flow Rate Verification (QC check) PM 10 (lo- Vol), PM 2.5 Once every 4 weeks <= 4% of Standard Flow Rate Verification (QC check) PM 10 (high- Vol), TSP Once per quarter <= 7% of Standard

19. 19 MethodPollutantsFrequencyMQO Collocated Sampling PM 10, TSP, PM 2.5 15% of Network Every 12 Days Precision as CV < = 10% (meaning the relative percent diff must be less than 14% for conc > 3 ug/m3)

20.  Keep the bias component minimized by calibrating and verifying your equipment against a standard  Keep precision component low by consistency  You can work to keep bias down, while precision is often out of your control below a certain limit  EPA calculates from your RA transactions 20

21. 21 MethodPollutantsFrequencyMQO Annual Performance Evaluation (Audit) SO 2, NO 2, O 3, CO Once per Year <= 15% for each audit concentration— OPERATIONAL Semi-Annual Flow Rate Audit PM 10, PM 2.5 Every 6 Months <= 4% of Standard OPERATIONAL Semi-Annual Flow Rate Audit PM 10 (high- Vol), TSP Every 6 Months <= 10% of Standard OPERATIONAL PM 2.5 PEP Program NPAP PM 2.5 SO 2, NO 2, O 3, CO Quarter Year (see QA Requireme nts.xls) Bias = 10% -- OPERATIONAL

22.  Bias: systematic difference, or “jump”  Precision: random error, or “wiggle”  Accuracy = total error, a combination of both  Simple temp bath illustrations at:  http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/AllDownloadableMovies/ http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/AllDownloadableMovies/ 22

23.  DASC (Data Assessment Statistical Calculator) ◦ http://itep68.itep.nau.edu/itep_download s/QA101_Resources/DASC%20EPA%20Prec %20and%20Bias%20Calculator/ http://itep68.itep.nau.edu/itep_download s/QA101_Resources/DASC%20EPA%20Prec %20and%20Bias%20Calculator/  AQS: Data Quality Indicators Report (AMP255) 23

24.  Minimize bias by regular calibrations  both accuracy (total error) and precision are first estimated by d i on an ongoing basis, by you  Audits verify total error estimates  Course website: http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/ http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/  Our emails: ◦ Bill Frietsche: frietsche.bill@epamail.epa.govfrietsche.bill@epamail.epa.gov ◦ Melinda.ronca-battista@nau.edu Melinda.ronca-battista@nau.edu ◦ Christopher.lee@nau.edu Christopher.lee@nau.edu 24

26. Appendix A to Part 58 Table A-2 of App A AQS TTN web site AQSP&A Spreadsheet Discoverer AMTIC web site AQS Helpline AQS AMP255 Data Quality Indicator Report

27. Appendix A to Part 58 Regulations that define QA reporting requirements for criteria pollutants Define assessments for each criteria pollutant 1 Pt QC check for gases Annual performance evaluation for gases Flow rate verification for particulate matter Semi-annual flow rate audit for particulate matter Collocated sampling requirements for particulate matter Pb audit strips for laboratory analysis QA Performance Evaluation Program for PM fine, PM coarse, and Pb Formerly called precision and accuracy data – still use these terms on AQS transactions

28. This document available at: http://itep68.itep.nau.edu/itep_downloads/QA101_Resources/session%204- prec%20&%20bias/ http://itep68.itep.nau.edu/itep_downloads/QA101_Resources/session%204- prec%20&%20bias/

29. AQS TTN web site http://www.epa.gov/ttn/airs/airsaqs/ http://www.epa.gov/ttn/airs/airsaqs/ AQSP&A Spreadsheet http://www.epa.gov/ttn/airs/airsaqs/padata/ http://www.epa.gov/ttn/airs/airsaqs/padata/ AQS Discoverer http://www.epa.gov/ttn/airs/airsaqs/aqsdiscover/ http://www.epa.gov/ttn/airs/airsaqs/aqsdiscover/ AMTIC web site http://www.epa.gov/ttn/amtic/ http://www.epa.gov/ttn/amtic/ AQS Help Line EPACALLCENTER@epa.gov EPACALLCENTER@epa.gov 1-866-411-4EPA (4372)

30. Example run of AMP255 shown onscreen

31.  All QC data helps EPA balance your costs of QC with needed information to protect health  Questions to AQS are welcomed  EPA committed to improving user friendliness  Course website: http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/ http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/  Our emails: ◦ Bill Frietsche: frietsche.bill@epamail.epa.govfrietsche.bill@epamail.epa.gov ◦ Melinda.ronca-battista@nau.edu Melinda.ronca-battista@nau.edu ◦ Christopher.lee@nau.edu Christopher.lee@nau.edu 31