1. what to compare against the validation templates (see templates in course webpage: http://itep68.itep.nau.edu/itep_downloads/QA101_ Resources/Validation%20Templates%20from%20Red book%20QA%20Handbook%20vol%20II.pdf http://itep68.itep.nau.edu/itep_downloads/QA101_ Resources/Validation%20Templates%20from%20Red book%20QA%20Handbook%20vol%20II.pdf ) 1

2.  April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp  April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) – Travis Maki  April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe  April 28: Calculating Bias and Precision and AQS reports – Angie Shatas  May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring  May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell  May 19: 40 CFR 58 App. A- PM filter and continuous methods –Brandy Toft 2

3.  Measuring  Comparing with MQO  Graphing it/ watching for its drift  Fixing it when needed Everything must be documented and, when significant, reported 3

4.  Instruments operating stably  Measure what you can, verify it is: ◦ Within limits ◦ Not slowly drifting up or down  Verify you are meeting your QAPP specs  QAPP specs assess bias and imprecision  Ideal is to estimate bias and imprecision, so that you can know and limit your overall error  Write it down! 4

5.  Preparing for the field  Checking your equipment ◦ Analyzer working, QC within limits ◦ Transfer standards also need to be working  Supporting stuff: ◦ Site, power, weather, frequency of your checks ok? Look to the future  Data recording system working ◦ Routine data ◦ QC data  Downloading the data 5

6. …the difference between your answer and the “truth” Two components of total error (accuracy):  Bias (jump)  Precision (wiggle) 6

7.  Minimized by calibrating your equipment against a standard ◦ Make sure the standard has TRACEABILITY to a NIST standard ◦ Compare result of the standard(s) to what your instruments read (pollutant, flow rate, temp, pressure) Measures any bias in your equipment at that time 7

8.  Some imprecision is unavoidable  Sometimes up, sometimes down– “random”  Estimated by measuring the same thing several times  Or by side-by-side collocated  Minimized by carefully following procedures 8

9.  The term “transfer” means that you are “transferring” the reliability of a measurement from a primary calibration standard to your field instrument  The primary calibration standard can be the place where you send your transfer standard to be recalibrated annually  Pressure, temperature, and flow rate of your instruments must be checked with your transfer standards 9

10.  For example, from CFR 50 Appendix L: “An authoritative flow rate standard shall be used for calibrating or verifying the sampler's flow rate measurement device with an accuracy of ±2 percent. The flow rate standard shall be a separate, stand-alone device... This flow rate standard must have its own certification and be traceable to a National Institute of Standards and Technology (NIST) primary standard for volume or flow rate. 10

11. 11

12.  Single-point QC check is when you conduct a check using one flow rate or concentration or value  1-point QC (used to be called span) is an example of a one-point check  Verifications DO NOT involve any changes to your equipment or settings  See list of QC checks in validation templates for each pollutant (see pg 184 of QA Handbook Vol II in course website http://itep68.itep.nau.edu/itep_downloads/QA101_R esources/ ) http://itep68.itep.nau.edu/itep_downloads/QA101_R esources/ 12

13.  Perform a check of your equipment: ◦ flow rate for PM ◦ concentration for gas ◦ internal mass for TEOM  This may be single-point (one flow rate) or multi-point (several flow rates)  If this is within specifications, record this and continue 13

14.  If results are NOT within specs, then the instrument must be adjusted  This adjustment means that the response of the instrument is changed, which is defined as a CALIBRATION  Calibrations must be multi-point  After calibrating, do QC check again 14

15.  Corrective Action if the flow rate difference between the transfer standard and the sampler is > 4%  Check sampler for internal and external leaks  Ensure that temp. and pressure are within normal ranges  Run check a 2 nd time  If still > 4%, perform a multipoint calibration followed by a single-point verification 15

16.  1-point QC check must show within 7% for O3, 10% for others (if starting out, every week)  Zero must not drift more than 2% of full scale  Span concentration must not drift more than 7% since last check 16

17. Gaseous Verifications  Gaseous instrument verifications can be conducted: ◦ Through-the-probe (is what you will be audited on—do this if possible) ◦ Through-the-manifold ◦ Through the back of the analyzer  To evaluate the sampling system it is recommended that at least some verifications be conducted through- the-probe 17

18. MethodPollutantsFrequencyMQO One-Point QCSO 2, NO 2, O 3, COEvery 2 Weeks O 3 : Precision = 7% Bias = 7% SO 2, NO 2, CO : Precision = 10% Bias = 10% Annual Performance Evaluation (Audit) SO 2, NO 2, O 3, COOnce per Year<= 15% for each audit concentration Flow Rate Verification (QC check) PM 10 (lo-Vol), PM 2.5 Once every 4 weeks <= 4% of Standard & 5% of Design Value Flow Rate Verification (QC check) PM 10 (high-Vol), TSP Once per quarter <= 7% of Standard & 10% of Design Value Semi-Annual Flow Rate Audit PM 10, PM 2.5 Every 6 Months <= 4% of Standard & 5% of Design Value Semi-Annual Flow Rate Audit PM 10 (high-Vol), TSP Every 6 Months <= 10% of Standard & 10% of Design Value Collocated SamplingPM 10, TSP, PM 2.5 15% of Network Every 12 Days PM 2.5, TSP, PM 10 : Precision = 10% PM 2.5 PEP Program NPAP PM 2.5 SO 2, NO 2, O 3, CO Quarter Year (see QA Requirements. xls) Bias = 10% 18

19. MethodPollutantsFrequencyWho? With What? One-Point QCSO 2, NO 2, O 3, CO Every 2 WeeksSite operatorAnnually calibration standard, owned by tribe Annual Performance Evaluation SO 2, NO 2, O 3, CO Once per YearNot site operator, no national certification required, not the same standard that is used for routine QC checks Flow Rate Verification (not required to be reported to AQS) PM 10, PM 2.5 Once every 4 weeks Can be site operator, no national certification required, not the same standards that are used to calibrate your analyzer Semi-Annual Flow Rate Audit PM 10, PM 2.5 Every 6 MonthsPreferably not site operator, no national certification required, not the same standards that are used for monthly QC checks PM PEP, NPAPPM 2.5, All gaseous See QA Requirements.xls OAQPS/TAMS certified auditor OAQPS/TAMS equipment 19

20. Gaseous Standards  All calibrations, verifications, and audits should be conducted with certified gas standards or a certified transfer standard (ozone).  Gas standards are certified by the vendor and can be recertified (EPA Protocol 600/R- 97/121)  Ozone transfer standards should be annually certified and traceable to a Standard Reference Photometer (SRP) 20

21.  DASC (Data Assessment Statistical Calculator) ◦ In Excel, easy to use ◦ Calculates what EPA calculates in AMP255 ◦ Can be used with any parameters ◦ Cites equation #s from CFR  AQS: Data Quality Indicators Report (AMP255) ◦ Covers all of the Requirements ◦ http://www.epa.gov/ttnamti1/qareport.ht ml http://www.epa.gov/ttnamti1/qareport.ht ml 21

22.  Consider the standard to be the “truth” for your equipment  Difference between the “truth” and your equipment is the instrument error (this may include bias [error either usually high or usually low], and imprecision [wiggle sometimes high and sometimes low])  How is instrument error quantified? 22

23. Basis for All Calculations (divide by best estimate of the truth, to get a %-off-from- truth):  For standard gas  For Collocated Samples 23

24. If you are outside the limits:  Review procedures & logs to identify problem  Go back and review data—may have to throw out (invalidate—never delete) data to the last passing check (see validation templates for critical criteria that require invalidation— may not have to invalidate data if it’s a systematic or operational criterion)  Fix the problem, document it and report (when significant) 24

25.  “Accuracy” for EPA means “total error”  Comparison conducted with completely different system can be used to estimate total error  For PM2.5, this is a performance evaluation conducted with a side-by-side FRM device, with the filter weighed by a different lab  Total error includes both precision and bias errors  Accuracy estimates are verified thru audits 25

26. Resources  AMTIC http://www.epa.gov/ttn/amtic/ http://www.epa.gov/ttn/amtic/  Transfer Standards for Calibration of Air Monitoring Analyzers for Ozone: Technical Assistance Document http://www.epa.gov/ttnamti1/files/ambient/qaqc/OzoneTransferStandardGuidance.p df  U. S. EPA Ambient Air Monitoring Protocol Gas Verification Program: Implementation Plan 2nd DRAFT 7/2009 http://www.epa.gov/ttnamti1/files/ambient/qaqc/aapgvpimpplan.pdf EPA NPAP TTP PE SOP COMPENDIUM (DRAFT FINAL, 1-12-05) http://www.epa.gov/ttn/amtic/files/ambient/npapsop/npa pttpsop.pdf 26

27. Resources  Field Standard Operating Procedures for the Federal PM2.5 Performance Evaluation Program http://www.epa.gov/ttn/amtic/files/ambient/pm25/qa/PEP_Field_S OP.pdf  Pb Performance Evaluation Program http://www.epa.gov/ttn/amtic/files/ambient/pb/Pb- PEPImplementationPlan_7_09.pdf  Model Quality Assurance Project Plan For the National Air Toxics Trends Stations http://www.epa.gov/ttn/amtic/files/ambient/airtox/NATTS_Model_ QAPP.pdf Resources—National QA Workgroup  QA National Meeting Presentations http://www.epa.gov/ttn/amtic/qaqcrein.html 27

28.  QC checks are necessary to both do, and  Track results of  Subsequent webinars will cover pollutant specifics QC checks  Course website: http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/ http://itep68.itep.nau.edu/itep_downloads/Q A101_Resources/  Our emails: ◦ Travis Maki: tmaki@itcmi.orgtmaki@itcmi.org ◦ Melinda.ronca-battista@nau.edu Melinda.ronca-battista@nau.edu ◦ Christopher.lee@nau.edu Christopher.lee@nau.edu 28