1. Air Quality Health Risk Assessment – Methodological Issues and Needs Presented to SAMSI September 19, 2007 Research Triangle Park, NC Anne E. Smith, Ph.D. Vice President asmith@crai.com (202) 662-3872

2. 2 Risk Assessment for Ambient Air Pollutants A policy analysis primarily performed by environmental protection agencies to assess benefits of air quality regulations –The analysis step where clinical and epidemiological evidence of “whether” there is a health effect shifts to asking “how much” is the public health being affected. Focus of this presentation is on issues in performing quantitative portion of an air quality risk assessment –Important not to lose sight of the qualitative elements of RA Typical quantitative risk assessment questions: –How much does ambient pollution affect public health outcomes? –How much would public health improve if we changed ambient air quality standards?

3. 3 Ambient Air Environmental Risk Assessment Part of formal USEPA air quality standards-setting process Frequently in the headlines

4. 4 Quantitative Risk Assessment Requires Extrapolations From epi associations  “dose-response” function From observed current ambient air  changes in “dose” # Health Events = f ( ΔExposure* Toxicity) Multiple Epi-derived Ests of Toxicity Current Ambient Concentration Distributions Changes due to Policy Policy “Benefits” Analysis of the added uncertainties is paramount …and a substantial challenge for analysts What is true toxicity value? What is true shape of f(.)? How will policy actually affect concentration distribution?

5. 5 Discussion of Uncertainties in the “Dose-Response” Function Portion of a Risk Assessment Epidemiological associations are quantitative, but that does not mean they should be interpreted at face value as “exposure-response functions”.

6. 6 Uncertainties in Using Epidemiological Associations for Extrapolating to Public Health Outcomes Are the associations evidence of a causal relationship? –Causality with respect to the specific ambient pollutant that the air quality standard applies to is especially uncertain –How should uncertainty on causality enter into risk estimates? Assuming causality: –Impact of exposure error on quantitative estimate of toxicity –Impact of unknown missing explanatory variables –Toxicity estimates from “multi-pollutant” vs. “single-pollutant” models –Unknown “correct” lag structure –Very limited capability of epidemiological estimates to identify any non-linearities in underlying concentration-response function ­Particularly important for extrapolation to much lower ambient concentration levels than studied

7. 7 Example: Multiple Estimates in Epidemiological Literature on Chronic Exposure Mortality Risk from Fine Particulate Matter (PM 2.5 ) 3 Cohorts Studied “Am Cancer Institute” “6 Cities” “Veteran’s Admin” “1-pollutant” formulation “2-pollutant” formulation “1-pollutant” formulation “2-pollutant” formulation “1-pollutant” formulation , (σ  ).005 (.001).006 (.002).006 (.003).006 (.004).004 (.002).002 (.001) , (σ  ).003 (.002).012 (.004).004 (.004).002 (.004) , (σ  ).013 (.004).008 (.004) For details of analysis: Anne Smith, Comments to USEPA, 3/31/07 , (σ  ) <0 (signif) <0 (insignif) , (σ  ) <0 (signif) Single estimate used in USEPA’s deterministic risk assessment

8. 8 0-2%2-4% 4-6% 6-8% 8-10% 10-12%12-14%14-16%16-18%18-20%20-22%22-24% 24-26% No PM2.5 risk at all Estimates of percent of long-term mortality incidence attributed to PM 2.5 -- Los Angeles for 2003 ambient levels – USEPA RA’s 95% confidence interval based on single regression Example of Results of Integrated Uncertainty Analysis for PM 2.5 Mortality, Compared to Deterministic Results For documentation of Integrated Uncertainty Analysis assumptions, see “Appendix C” of Anne Smith’s Comments on the Second Draft of EPA’s PM 2.5 Risk Analysis, 3/31/05 (available in PMDocket or on request from author). EPA RA estimates are from Final Risk Assessment (July 2005) = 3684 deaths per year in Los Angeles due to 2003 PM 2.5 concentrations

9. 9 Example: Multiple Estimates in Epidemiological Literature on Chronic Exposure Mortality Risk from Fine Particulate Matter (PM 2.5 ) 3 Cohorts Studied “Am Cancer Institute” “6 Cities” “Veteran’s Admin” “1-pollutant” formulation “2-pollutant” formulation “1-pollutant” formulation “2-pollutant” formulation “1-pollutant” formulation , (σ  ).005 (.001).006 (.002).006 (.003).006 (.004).004 (.002).002 (.001) , (σ  ).003 (.002).012 (.004).004 (.004).002 (.004) , (σ  ).013 (.004).008 (.004) For details of analysis: Anne Smith, Comments to USEPA, 3/31/07 , (σ  ) <0 (signif) <0 (insignif).33 , (σ  ) <0 (signif) Single estimate used in USEPA’s deterministic risk assessment.5 Illustrative assignments of weights for uncertainty analysis

10. 10 0% 10% 20% 30% 40% 50% 60% 0-2%2-4% 4-6% 6-8% 8-10% 10-12%12-14%14-16%16-18%18-20%20-22%22-24% 24-26% No PM2.5 risk at all Estimates of percent of long-term mortality incidence attributed to PM 2.5 -- Los Angeles for 2003 ambient levels – USEPA RA’s 95% confidence interval based on single regression Example of Results of Integrated Uncertainty Analysis for PM 2.5 Mortality, Compared to Deterministic Results For documentation of Integrated Uncertainty Analysis assumptions, see “Appendix C” of Anne Smith’s Comments on the Second Draft of EPA’s PM 2.5 Risk Analysis, 3/31/05 (available in PMDocket or on request from author). EPA RA estimates are from Final Risk Assessment (July 2005) Histogram of pdf from Illustrative Integrated Uncertainty Analysis

11. 11 Discussion of Uncertainties in the Exposure Portion of a Risk Assessment

12. 12 Uncertainties in Pollutant Exposure Changes due to Policy: The Pollutant “Rollback” Assumption 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0369 121518 212427303336394245 4851 5457 606366 6972 Daily 8-Hour Maxima (PPB) Cumulative Frequency Key RA assumption: how each part of the pollutant exposure distribution would be affected by a change in the ambient standard Large share of change assumed to occur on days with relatively low pollutant concentration (e.g., well below the standard) -- Rollback Alternative 1 --

13. 13 Uncertainties in Pollutant Exposure Changes due to Policy: The Pollutant “Rollback” Assumption 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 0369 121518 212427303336394245 4851 5457 606366 6972 Daily 8-Hour Maxima (PPB) Cumulative Frequency -- Rollback Alternative 2 -- Larger share of change assumed to occur on days with relatively high pollutant levels Rollback if there is a substantial “background” or uncontrollable portion of pollutant along the distribution

14. 14 Needs for Methodological Development

15. 15 Bottom Line of these Examples & Research Needs Health risk assessment for air quality standards is subject to enormous uncertainty Key uncertainties are not statistical –Model selection and model shape for exposure-response –Modeling of air quality distributional changes as result of policy Initial examples of methods to incorporate the above forms of uncertainty into the standard risk assessments produce very different characterization of risk and of benefits from air quality standards The challenge to the risk analysis community is in developing workable approaches for –Integrating multiple sources of uncertainty –Dealing with the highly judgmental aspects required to represent the key sources of these uncertainties

16. 16 Coda: Estimated Annual “Lives Saved” from Revised PM 2.5 Ambient Standard (as published with Final Rule) Some use of uncertainty analysis as rule was finalized -- based on direct elicitation -- did not address any uncertainties in exposure (“rollbacks”) -- still many issues to be addressed …and other pollutants (e.g., ozone) EPA’s Regulatory Impact Assessment, p. ES-8:

17. 17 Top-down method “Direct Elicitation”: Ask “experts” what they believe to be likelihood of different levels of pollutant’s risk, including their personal uncertainty. Bottom-up method, “Evidence-Driven Elicitation”: Ask experts to assign weights to all relevant epidemiological model results, based on their personal views about the relative quality of each model formulation/study design, etc. Two Approaches for Characterizing Probability Distributions on Concentration-Response Relationships Disadvantages in the epi-based risk analysis situation: –The vast majority of “experts” are those who wrote the epidemiological papers (motivational bias) –By the time risk analysis is starting, most potential experts are viewed as having a “position” on the decisions that the risk analysis will inform Advantages in the epi-based risk analysis situation: –Experts can be disinterested outsiders to the study of the pollutant in question; need to be experts only in their ability to interpret regression-based studies generally –“Evidence-driven”: creates explicit linkage of weights to subjective opinions about relative quality of underlying studies –Explicit criteria for “quality” in a model can be articulated –It is possible to “blind” the experts to the quantitative estimates in each study (minimization of potential motivational bias)