Presentation is loading. Please wait.

Presentation is loading. Please wait.

Freshwater Sediment Standards Science Panel August 25, 2010 Presenters: Russ McMillan – Department of Ecology Teresa Michelsen – Avocet Consulting 1.

Similar presentations


Presentation on theme: "Freshwater Sediment Standards Science Panel August 25, 2010 Presenters: Russ McMillan – Department of Ecology Teresa Michelsen – Avocet Consulting 1."— Presentation transcript:

1 Freshwater Sediment Standards Science Panel August 25, 2010 Presenters: Russ McMillan – Department of Ecology Teresa Michelsen – Avocet Consulting 1

2 Freshwater Sediment Standards Goal For Today Are the approaches used to develop chemical and biological criteria scientifically defensible? 2

3 Freshwater Sediment Standards Discussion Points For Today Introduce regulatory and policy context Introduce regulatory and policy context Present proposed biological and chemical criteria and framework Present proposed biological and chemical criteria and framework Identify and discuss scientific and technical issues Identify and discuss scientific and technical issues 3

4 Freshwater Sediment Standards Policy Decisions Consistency with SMS regulatory framework. Consistency with SMS regulatory framework. Adopt biological and chemical criteria. Adopt biological and chemical criteria. Two tier structure: SQS and CSL. Two tier structure: SQS and CSL. Allowance of some adverse effects. Allowance of some adverse effects. Biological override of chemical criteria. Biological override of chemical criteria. 4

5 Confirmatory bioassays override chemistry Confirmatory bioassays override chemistry Two tier structure: SQS and CSL Two tier structure: SQS and CSL Bioassay suite – Multiple species/sensitive life-history stages Bioassay suite – Multiple species/sensitive life-history stages Minimum of 3 endpoints Minimum of 3 endpoints Both acute and chronic tests Both acute and chronic tests Proposed Biological Sediment Standards Regulatory Framework 5

6 Narrower choice of species than SMS marine criteria due to limited bioassay availability. Narrower choice of species than SMS marine criteria due to limited bioassay availability. Built on EPA and ASTM protocols well established in the Northwest: Built on EPA and ASTM protocols well established in the Northwest: Hyalella 10-day mortality – 366 Hyalella 10-day mortality – 366 Hyalella 28-day mortality – 312 Hyalella 28-day mortality – 312 Hyalella 28-day growth – 79 Hyalella 28-day growth – 79 Chironomus 10-day growth – 525 Chironomus 10-day growth – 525 Chironomus 10-day mortality – 568 Chironomus 10-day mortality – 568 Proposed Suite of Bioassays 6

7 Consistent w/SMS designation of sediment quality Consistent w/SMS designation of sediment quality SQS: Single SQS level hit SQS: Single SQS level hit CSL: 2+ SQS level hits; 1+ CSL level hit CSL: 2+ SQS level hits; 1+ CSL level hit Consistent w/SMS sensitive endpoints. Consistent w/SMS sensitive endpoints. Proposed Suite of Bioassays 7

8 Bioassay suite will include sensitive endpoints from chronic and acute tests: 3 Endpoints 3 Endpoints 2 Species 2 Species 1 Chronic test 1 Chronic test 1 Sublethal endpoint 1 Sublethal endpoint Requirements for Proposed Bioassay Suite 8

9 Test Acute Bioassays Chronic Bioassays Lethal Endpoint Sub-lethal Endpoint Hyalella azteca 10-day mortalityX X 28-day mortality XX 28-day growth X X Chironomus dilutus 10-day mortalityX X 10-day growthX X 20-day mortality XX 20-day growth X X Bioassays: Acute and Chronic and Endpoint Effects Levels 9

10 Is the proposed bioassay suite scientifically defensible as being appropriately protective of the benthic community? Proposed Bioassay Suite Question 10

11 Differs from SMS marine interpretation due to difficulty in identifying appropriate reference areas. Differs from SMS marine interpretation due to difficulty in identifying appropriate reference areas. Due to lack of reference sites, interpretation of an SQS and CSL hit is based on a comparison to control. Due to lack of reference sites, interpretation of an SQS and CSL hit is based on a comparison to control. Comparison to control is a more conservative interpretation than comparison to a reference sediment. Comparison to control is a more conservative interpretation than comparison to a reference sediment. Proposed Freshwater Bioassay Interpretation 11

12 Test QA limits Control QA limits Reference SQSCSL Hyalella azteca 10-day mortality C 20%R 25% T – C > 15%T – C > 25% 28-day mortality C 20%R 30% T – C > 10%T – C > 25% 28-day growth CF 0.15 mg/ind RF 0.15 mg/ind T/C < 0.75T/C < 0.6 Chironomus dilutus 10-day mortality C 30%R 30% T – C > 20%T – C > 30% 10-day growth CF 0.48 mg/ind RF/CF 0.8 T/C < 0.8T/C < day mortality C 32%R 35% T – C > 15%T – C > 25% 20-day growth CF 0.48 mg/ind RF/CF 0.8T/C < 0.75T/C < 0.6 C=Control, R=Reference, T=Test, F=Final, (SQS &CSL hits statistically sign. diff.) Bioassay Interpretation: Comparison to Control 12

13 Is it scientifically defensible to base interpretation of a bioassay hit by using a comparison to control rather than a comparison to a reference sediment? Is it scientifically defensible to base the designation of sediment quality on a suite of bioassays comparing test to control without the benefit of a reference sediment? Question 13

14 14

15 History of Freshwater SQV Development Early work on FW Apparent Effects Thresholds (AETs) & Floating Percentile Method (FPM; Portland Harbor) throughout the late 1990s Early work on FW Apparent Effects Thresholds (AETs) & Floating Percentile Method (FPM; Portland Harbor) throughout the late 1990s 2002 – Formal evaluation of FW AETs and other existing SQV sets (TELs/PELs, etc.) 2002 – Formal evaluation of FW AETs and other existing SQV sets (TELs/PELs, etc.) Decision that a new approach was needed: Decision that a new approach was needed: FW AETs not sufficiently conservative FW AETs not sufficiently conservative TELs/PELs, etc., far too conservative TELs/PELs, etc., far too conservative National evaluations were not looking at both types of statistical errors National evaluations were not looking at both types of statistical errors 15

16 Statistical Digression False Negative (FN) = Predicting that a sample will be non-toxic when it is actually toxic False Negative (FN) = Predicting that a sample will be non-toxic when it is actually toxic False Positive (FP) = Predicting that a sample will be toxic when it is actually non-toxic False Positive (FP) = Predicting that a sample will be toxic when it is actually non-toxic Existing national methods were focused on reducing false negatives at lower screening levels and false positives at upper screening levels, creating substantial errors and inefficiencies in between, where most actual data are located. We focused on reducing both types of errors at the same time, for all levels of effects. 16

17 History, cont – Developed interim FPM values, used as guidance by Ecology and in the regional dredging manual (SEF) 2003 – Developed interim FPM values, used as guidance by Ecology and in the regional dredging manual (SEF) 2007 – Regional Sediment Evaluation Team (RSET) OR/WA workgroup formed to update FPM SQVs 2007 – Regional Sediment Evaluation Team (RSET) OR/WA workgroup formed to update FPM SQVs 2008 – RSET technical work completed; final SQV selection still under discussion and review 2008 – RSET technical work completed; final SQV selection still under discussion and review 2009/2010 – Ecology begins rule revision, finalizes the values, begins peer review 2009/2010 – Ecology begins rule revision, finalizes the values, begins peer review 17

18 Past and current review Presentations at 5 national and regional scientific conferences ( ) Presentations at 5 national and regional scientific conferences ( ) DEQ-led peer review & public meetings during Portland Harbor (2001 state site) DEQ-led peer review & public meetings during Portland Harbor (2001 state site) Public and agency review of 2003 Ecology report Public and agency review of 2003 Ecology report Presentations at 4 SMARMs ( ) + numerous RSET public meetings Presentations at 4 SMARMs ( ) + numerous RSET public meetings SMS Advisory Workgroup peer review of approach and draft SQV report (2010) SMS Advisory Workgroup peer review of approach and draft SQV report (2010) Science Panel and national peer review (2010) Science Panel and national peer review (2010) 18

19 Projects/Guidance to Date 1999, 2003, 2008 Portland Harbor, OR 1999, 2003, 2008 Portland Harbor, OR 2001 Onondaga Lake, NY 2001 Onondaga Lake, NY 2003 Los Angeles Harbor, CA 2003 Los Angeles Harbor, CA 2004 San Francisco Bay/Oakland Harbor, CA 2004 San Francisco Bay/Oakland Harbor, CA 2003 Draft Ecology Freshwater Guidelines, WA 2003 Draft Ecology Freshwater Guidelines, WA 2006 Interim Sediment Evaluation Framework Guidelines for WA/OR/ID 2006 Interim Sediment Evaluation Framework Guidelines for WA/OR/ID 2010 Updated Freshwater Guidelines, WA 2010 Updated Freshwater Guidelines, WA 19

20 Question 1 General Approach: - Is it appropriate to use sediment bioassays to represent effects to the benthic community? - Is the use of a multivariate model to empirically derive chemical SQVs scientifically defensible? 20

21 Use of toxicity tests Marine AETs were derived based on toxicity tests, benthic community studies, and chemistry Marine AETs were derived based on toxicity tests, benthic community studies, and chemistry Benthic community studies were considered equivalent to a chronic bioassay Benthic community studies were considered equivalent to a chronic bioassay Freshwater benthic community data were searched for but not found in OR, WA, or ID Freshwater benthic community data were searched for but not found in OR, WA, or ID Substantial diversity of freshwater sites compared to marine areas complicates use of benthic community data, if there were any Substantial diversity of freshwater sites compared to marine areas complicates use of benthic community data, if there were any 21

22 Floating Percentile Method Goal: Minimize false negatives and false positives simultaneously Approach: Data QA, screening, and summing Determine true toxicity based on bioassay results Search for the most predictive chemical concentrations, allowing each chemical to move independently to the level at which it appears to be toxic 22

23 Features Uses synoptic chemistry and bioassay field data Uses synoptic chemistry and bioassay field data Multivariate considers all chemicals at once Multivariate considers all chemicals at once Incorporates measures to address covariance Incorporates measures to address covariance Requires selection of false negative target Requires selection of false negative target Follows with optimization of false positives Follows with optimization of false positives Repeat for a range of false negative targets Repeat for a range of false negative targets Thoroughly evaluates reliability Thoroughly evaluates reliability Now automated using a series of Excel spreadsheets Now automated using a series of Excel spreadsheets 23

24 24

25 Covariance All chemicals are addressed simultaneously – avoids inappropriate assignment of toxicity All chemicals are addressed simultaneously – avoids inappropriate assignment of toxicity Covariance analysis can be run ahead of time Covariance analysis can be run ahead of time Model results allow visual identification of covariance patterns Model results allow visual identification of covariance patterns Appropriate chemical classes can be summed Appropriate chemical classes can be summed Those that cant be summed but often covary are subjected to multiple runs with different starting points to find the low concentration for each chemical, then selection of the combination with the highest reliability Those that cant be summed but often covary are subjected to multiple runs with different starting points to find the low concentration for each chemical, then selection of the combination with the highest reliability 25

26 Question 2 Data Issues: - Is the data set sufficiently robust and representative? - Has appropriate data screening and QA been conducted? 26

27 Data Set – Chemistry Oregon and Washington Oregon and Washington West and east of the Cascade Mountains West and east of the Cascade Mountains Lakes, rivers, small and large Lakes, rivers, small and large Various geochemical environments Various geochemical environments 50 analytes and sums 105 chemicals 50 analytes and sums 105 chemicals Rigorous QA/QC applied (PSEP QA2) – qualifiers rectified among data sets Rigorous QA/QC applied (PSEP QA2) – qualifiers rectified among data sets 27

28 Data Set – Bioassay Endpoints Hyalella 10-day mortality – 366 Hyalella 10-day mortality – 366 Chironomus 10-day mortality – 550 Chironomus 10-day mortality – 550 Chironomus 10-day growth – 504 Chironomus 10-day growth – 504 Hyalella 28-day mortality – 319 Hyalella 28-day mortality – 319 Hyalella 28-day growth – 79 Hyalella 28-day growth – 79 Rigorous QA/QC applied – recent ASTM protocols, QA2 review of lab sheets, etc. Rigorous QA/QC applied – recent ASTM protocols, QA2 review of lab sheets, etc. 28

29 Question 3 Reliability Testing: - Is the reliability testing that was conducted an appropriate method for evaluating SQVs? - Is the comparative reliability analysis that was conducted an appropriate way of making decisions while fine-tuning the approach? - Are the reliability measures that were used the right ones and were the relative weights given to them appropriate? - Are there alternative methods of validation that could have been used without collecting additional data? 29

30 Reliability Sensitivity (100% – false negatives) Sensitivity (100% – false negatives) Efficiency (100% – false positives) Efficiency (100% – false positives) Predicted no-hit reliability Predicted no-hit reliability Predicted hit reliability Predicted hit reliability Overall reliability Overall reliability All measures of reliability were used for ALL effects levels – endpoints given greater weight are shown in yellow 30

31 Reliability Measures Diagram 31

32 Reliability Goals The RSET Workgroup set the following goals before beginning SQV development: SQS (%)CSL (%) Sensitivity80 – 9075 – 85 Efficiency70 – 8075 – 85 Predicted Hit Reliability 70 – 8075 – 85 Predicted No-Hit Reliability 80 – 9075 – 85 Overall Reliability80 – 90 32

33 33

34 Freshwater Standards Reliability Values are averages across relevant assays No Effect Level Minor Effect Level 34

35 Comparative Reliability Analysis East side vs. west side vs. combined East side vs. west side vs. combined TPH vs. PAH vs. combined TPH vs. PAH vs. combined Microtox – include? Microtox – include? Hyalella growth – include Portland Harbor? Hyalella growth – include Portland Harbor? Ammonia and sulfides issues Ammonia and sulfides issues N-qualified pesticides N-qualified pesticides Blank-correction standardization Blank-correction standardization Comparison to control vs. reference Comparison to control vs. reference 35

36 SQV Validation RSET made a decision early on to not withhold part of the data set for validation RSET made a decision early on to not withhold part of the data set for validation For such a large heterogeneous area, we needed all the data to develop the best possible model For such a large heterogeneous area, we needed all the data to develop the best possible model Most other SQV sets have followed the same approach, with independent validation following after Most other SQV sets have followed the same approach, with independent validation following after Independent validation will require a large, representative data set, not just a few projects Independent validation will require a large, representative data set, not just a few projects Other validation methods available? Other validation methods available? 36

37 Question 4 Data Interpretation and Regulatory Decision- Making: - Is the overall framework and selection of final SQVs consistent with the SMS and marine SQVs? - Is the approach used to select final SQVs scientifically defensible? 37

38 Challenges – Criteria Selection Expectation: SQS values clustered below CSL values SQS CSL SQS CSL 38

39 Challenges – Criteria Selection Reality: Differences between bioassays were much greater than differences between endpoints try species sensitivity distribution approach 39

40 > values- no toxicity observed for that endpoint up to the listed concentration. Sample concentrations at or above this level should undergo toxicity testing. > values- no toxicity observed for that endpoint up to the listed concentration. Sample concentrations at or above this level should undergo toxicity testing. Approach for selection of CSL: next significantly different value Approach for selection of CSL: next significantly different value 40

41 Questions? 41

42 Petroleum Toxicity PAHs contribute to the greatest number of errors: PAHs do not sufficiently capture petroleum toxicity PAHs do not sufficiently capture petroleum toxicity No form of normalizing or summing solves the problem No form of normalizing or summing solves the problem Addressed through side-by-side PAH/TPH & combined model runs Addressed through side-by-side PAH/TPH & combined model runs Best results when both were included Best results when both were included May be legacy issues with not enough TPH data May be legacy issues with not enough TPH data 42

43 Summation of Chemical Classes PAHs/TPH classes PAHs/TPH classes PCB Aroclors PCB Aroclors Dioxins/Furans Dioxins/Furans Chlordanes Chlordanes DDT, DDE, DDD isomers DDT, DDE, DDD isomers Heptachlors Heptachlors 43


Download ppt "Freshwater Sediment Standards Science Panel August 25, 2010 Presenters: Russ McMillan – Department of Ecology Teresa Michelsen – Avocet Consulting 1."

Similar presentations


Ads by Google