1. Module 6: Uncertainty Don’t just calculate—first think about sources of error, and don’t double-count errors
Presentation of Uncertainty:
1. Components of overall uncertainty (field and lab measurement errors, representativeness)
2. Confidence intervals
3. Graphical presentation and interpretation
module 6

2. Two Sources of Error Sampling Population
How/where/when/who makes the measurements
Population
Actual variability in what you are measuring
module 6

3. Measurement error specific to…
Operator
Instrument
Lab
Procedure
Standard
Time (day of week, year, season)
Measurement level (harder to measure at low concentrations)
module 6

4. Population Error
Ideally, estimate some aspect of homogenous “clump” of air, water, people
If population is totally homogenous, only one measurement is necessary
The more variability in the population, the more measurements you need
module 6

5. Minimizing the effect of population uncertainty
Careful sampling plan, designed to include measurements from all “over” the distribution
Sampling plan to measure smallest “homogenous” parts of environment as possible
Careful adherence to identical procedures
module 6

6. QC measurements designed to…
Identify where errors occur
Quantify errors (difference from “reality”)
Save $ by improving program
Produce estimates of how certain your conclusions can be…
…therefore allowing decisions based on what you really “can” know
Discuss diff parts of a QC system, with checks for lab, checks rotating around diff operators and sites, etc.
module 6

7. PM QC Results Collocated Flow rate checks done with routine standard
Flow rate checks done with an external standard
PEP intercomparisons of external instrument and lab
What to do with each?
module 6

8. How EPA Summarizes QC First, estimate uncertainty for each site
Use collocated results to calculate confidence interval for precision (CV)
Start with RPD (diff/mean)
Always use same pair and order
module 6

9. See P&B DASC with PM Data
This is brandy’s data that we will use for the calcs in the DASC.xls file
module 6

10. PM2.5 Precision Estimate (40 CFR 58 App. A eq’n 11)
This is the equation from CFR (eqn 11 in appendix A) that is the calculation of the upper confidence limit for the precision with a 90% level of confidence. This means that for this time period we can be sure that 90% of the time, if these results really represented the imprecision in the whole system, that 90% of the collocated results would be less than about 7.7%
module 6

11. 90% confidence limit for precision = 7.7%
Average over quarter = x microg/m3 +- 8% (with 90% confidence, from precision error alone)
Can use this as part of overall uncertainty estimate
Combine with bias estimates from flow rate and PEP audits
Use this in tracking QC results: are results in next qrtr better or worse?
module 6

12. To estimate bias… Use PEP audit results, if available
Use any comparisons that are independent as possible
Use DASC PM2.5 Bias (Current PEP) tab
Calculates upper and lower 90% confidence intervals
module 6

13. UCL and LCL (upper and lower confidence level)
module 6

14. What does this mean? UCL is ~ 10% LCL is ~ -10%
Uncertainty of bias about 10%
Average bias of 7% could really be 7.7, or about 8%
module 6

15. Combining precision and bias?
For rough estimate: square root of sum of squares
Start with d=diff/mean for all QC checks
Calc STDEV of each set of d’s
Square each STDEV
Add squares
Take square root, see if it makes sense!
module 6

16. Precision for qrtr 1 of 2003
Collocated pairs, so PRECISION estimate (if A is not consistently higher/lower than B)
Results of collocated pairs, the stdev of the d’s (RPD between pairs)
This is going to be less than the 90% CL for precision.
module 6

17. Bias for 2003, based on independent checks
module 6

18. Square Root of Sum of Squares
HOORAY THIS MAKES SENSE! An estimate of total error of about 11% makes sense: the mean, plus or minus 11%, will contain the true mean more than half the time….(cant get too specific here, it is a rough estimate).
module 6

19. Presenting Uncertainty
Use error bars or upper, lower lines in graph
In exercise we’ll do more sophisticated graphs
module 6

20. Uncertainty for Gaseous Methods
Simpler than PM
RPD between known and measured for automated and manual checks
Estimates validated with results of independent audits
QC checks produce estimates that include both precision and bias error
module 6

21. Summarizing Uncertainty: COMMON SENSE first!
“Highest” estimate or worst-case calculation from results of independent audits (that encompass both precision and bias)
Uncertainty estimate should encompass (already include the error from) your internal assessments, so do not double-count results
module 6