what to compare against the validation templates (see templates in course webpage: Resources/Validation%20Templates%20from%20Red book%20QA%20Handbook%20vol%20II.pdf Resources/Validation%20Templates%20from%20Red book%20QA%20Handbook%20vol%20II.pdf ) 1

 April 7: QA Systems, EPA definitions, PQAOs and common sense – Mike Papp  April 14: Routine Quality Control and Data Management (1-pt QC, flow rate, and instrument stability checks) – Travis Maki  April 21: Audits Overview (NPAP, PEP, Annual PE, Flow Rate Audits) – Jeremy Howe  April 28: Calculating Bias and Precision and AQS reports – Angie Shatas  May 5: 40 CFR 58 App. A- Gaseous Pollutants – Glenn Gehring  May 12: 40 CFR 58 App. A- Ozone – Brenda Jarrell  May 19: 40 CFR 58 App. A- PM filter and continuous methods –Brandy Toft 2

 Measuring  Comparing with MQO  Graphing it/ watching for its drift  Fixing it when needed Everything must be documented and, when significant, reported 3

 Instruments operating stably  Measure what you can, verify it is: ◦ Within limits ◦ Not slowly drifting up or down  Verify you are meeting your QAPP specs  QAPP specs assess bias and imprecision  Ideal is to estimate bias and imprecision, so that you can know and limit your overall error  Write it down! 4

 Preparing for the field  Checking your equipment ◦ Analyzer working, QC within limits ◦ Transfer standards also need to be working  Supporting stuff: ◦ Site, power, weather, frequency of your checks ok? Look to the future  Data recording system working ◦ Routine data ◦ QC data  Downloading the data 5

…the difference between your answer and the “truth” Two components of total error (accuracy):  Bias (jump)  Precision (wiggle) 6

 Minimized by calibrating your equipment against a standard ◦ Make sure the standard has TRACEABILITY to a NIST standard ◦ Compare result of the standard(s) to what your instruments read (pollutant, flow rate, temp, pressure) Measures any bias in your equipment at that time 7

 Some imprecision is unavoidable  Sometimes up, sometimes down– “random”  Estimated by measuring the same thing several times  Or by side-by-side collocated  Minimized by carefully following procedures 8

 The term “transfer” means that you are “transferring” the reliability of a measurement from a primary calibration standard to your field instrument  The primary calibration standard can be the place where you send your transfer standard to be recalibrated annually  Pressure, temperature, and flow rate of your instruments must be checked with your transfer standards 9

 For example, from CFR 50 Appendix L: “An authoritative flow rate standard shall be used for calibrating or verifying the sampler's flow rate measurement device with an accuracy of ±2 percent. The flow rate standard shall be a separate, stand-alone device... This flow rate standard must have its own certification and be traceable to a National Institute of Standards and Technology (NIST) primary standard for volume or flow rate. 10

11

 Single-point QC check is when you conduct a check using one flow rate or concentration or value  1-point QC (used to be called span) is an example of a one-point check  Verifications DO NOT involve any changes to your equipment or settings  See list of QC checks in validation templates for each pollutant (see pg 184 of QA Handbook Vol II in course website esources/ ) esources/ 12

 Perform a check of your equipment: ◦ flow rate for PM ◦ concentration for gas ◦ internal mass for TEOM  This may be single-point (one flow rate) or multi-point (several flow rates)  If this is within specifications, record this and continue 13

 If results are NOT within specs, then the instrument must be adjusted  This adjustment means that the response of the instrument is changed, which is defined as a CALIBRATION  Calibrations must be multi-point  After calibrating, do QC check again 14

 Corrective Action if the flow rate difference between the transfer standard and the sampler is > 4%  Check sampler for internal and external leaks  Ensure that temp. and pressure are within normal ranges  Run check a 2 nd time  If still > 4%, perform a multipoint calibration followed by a single-point verification 15

 1-point QC check must show within 7% for O3, 10% for others (if starting out, every week)  Zero must not drift more than 2% of full scale  Span concentration must not drift more than 7% since last check 16

Gaseous Verifications  Gaseous instrument verifications can be conducted: ◦ Through-the-probe (is what you will be audited on—do this if possible) ◦ Through-the-manifold ◦ Through the back of the analyzer  To evaluate the sampling system it is recommended that at least some verifications be conducted through- the-probe 17

MethodPollutantsFrequencyMQO One-Point QCSO 2, NO 2, O 3, COEvery 2 Weeks O 3 : Precision = 7% Bias = 7% SO 2, NO 2, CO : Precision = 10% Bias = 10% Annual Performance Evaluation (Audit) SO 2, NO 2, O 3, COOnce per Year<= 15% for each audit concentration Flow Rate Verification (QC check) PM 10 (lo-Vol), PM 2.5 Once every 4 weeks <= 4% of Standard & 5% of Design Value Flow Rate Verification (QC check) PM 10 (high-Vol), TSP Once per quarter <= 7% of Standard & 10% of Design Value Semi-Annual Flow Rate Audit PM 10, PM 2.5 Every 6 Months <= 4% of Standard & 5% of Design Value Semi-Annual Flow Rate Audit PM 10 (high-Vol), TSP Every 6 Months <= 10% of Standard & 10% of Design Value Collocated SamplingPM 10, TSP, PM % of Network Every 12 Days PM 2.5, TSP, PM 10 : Precision = 10% PM 2.5 PEP Program NPAP PM 2.5 SO 2, NO 2, O 3, CO Quarter Year (see QA Requirements. xls) Bias = 10% 18

MethodPollutantsFrequencyWho? With What? One-Point QCSO 2, NO 2, O 3, CO Every 2 WeeksSite operatorAnnually calibration standard, owned by tribe Annual Performance Evaluation SO 2, NO 2, O 3, CO Once per YearNot site operator, no national certification required, not the same standard that is used for routine QC checks Flow Rate Verification (not required to be reported to AQS) PM 10, PM 2.5 Once every 4 weeks Can be site operator, no national certification required, not the same standards that are used to calibrate your analyzer Semi-Annual Flow Rate Audit PM 10, PM 2.5 Every 6 MonthsPreferably not site operator, no national certification required, not the same standards that are used for monthly QC checks PM PEP, NPAPPM 2.5, All gaseous See QA Requirements.xls OAQPS/TAMS certified auditor OAQPS/TAMS equipment 19

Gaseous Standards  All calibrations, verifications, and audits should be conducted with certified gas standards or a certified transfer standard (ozone).  Gas standards are certified by the vendor and can be recertified (EPA Protocol 600/R- 97/121)  Ozone transfer standards should be annually certified and traceable to a Standard Reference Photometer (SRP) 20

 DASC (Data Assessment Statistical Calculator) ◦ In Excel, easy to use ◦ Calculates what EPA calculates in AMP255 ◦ Can be used with any parameters ◦ Cites equation #s from CFR  AQS: Data Quality Indicators Report (AMP255) ◦ Covers all of the Requirements ◦ ml ml 21

 Consider the standard to be the “truth” for your equipment  Difference between the “truth” and your equipment is the instrument error (this may include bias [error either usually high or usually low], and imprecision [wiggle sometimes high and sometimes low])  How is instrument error quantified? 22

Basis for All Calculations (divide by best estimate of the truth, to get a %-off-from- truth):  For standard gas  For Collocated Samples 23

If you are outside the limits:  Review procedures & logs to identify problem  Go back and review data—may have to throw out (invalidate—never delete) data to the last passing check (see validation templates for critical criteria that require invalidation— may not have to invalidate data if it’s a systematic or operational criterion)  Fix the problem, document it and report (when significant) 24

 “Accuracy” for EPA means “total error”  Comparison conducted with completely different system can be used to estimate total error  For PM2.5, this is a performance evaluation conducted with a side-by-side FRM device, with the filter weighed by a different lab  Total error includes both precision and bias errors  Accuracy estimates are verified thru audits 25

Resources  AMTIC  Transfer Standards for Calibration of Air Monitoring Analyzers for Ozone: Technical Assistance Document df  U. S. EPA Ambient Air Monitoring Protocol Gas Verification Program: Implementation Plan 2nd DRAFT 7/ EPA NPAP TTP PE SOP COMPENDIUM (DRAFT FINAL, ) pttpsop.pdf 26

Resources  Field Standard Operating Procedures for the Federal PM2.5 Performance Evaluation Program OP.pdf  Pb Performance Evaluation Program PEPImplementationPlan_7_09.pdf  Model Quality Assurance Project Plan For the National Air Toxics Trends Stations QAPP.pdf Resources—National QA Workgroup  QA National Meeting Presentations 27

 QC checks are necessary to both do, and  Track results of  Subsequent webinars will cover pollutant specifics QC checks  Course website: A101_Resources/ A101_Resources/  Our s: ◦ Travis Maki: ◦ ◦ 28