1. Statistical Methods for Assessing Compliance – case studies Task 3.1B
Anja Duffek, Hannah Green, Markus Lehmann, John Batty

2. CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty
Task 3.1B
Background and aims of task
Results of the case studies
Discussion Points
Directive 2008/105/EC “… Member States may introduce statistical methods, such as a percentile calculation, to ensure an acceptable level of confidence and precision for determining compliance with the MAC-EQS.”
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

3. CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty
Task 3.1B – Background
Assessing compliance with MAC-EQS
(a) What type of monitoring regime?
Typically monthly (not continuous)
(b) What level of protection is in the standard?
Acute short-term exposure
Uncertainty
In the standard (e.g. variable assessment factors)
From intermittent monitoring
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

4. CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty
Task 3.1B – Background
Two philosophies regarding protection goals and what risk of wrong conclusions is acceptable :
High confidence of failure with small risk of false accusations (‘benefit of doubt’)
E.g. if action to secure compliance is expensive need to be confident it is necessary
At lower levels of confidence would increase monitoring frequency to improve understanding of environmental risk
Low confidence of failure with small risk of wrongly concluding compliant (‘fail safe’)
Highly protective approach
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

5. CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty
Task 3.1B – Progress
Paper to CEMP 25/11/10
Questionnaire on MAC compliance (April 2011)
Monitoring frequencies 4 – 12 samples per year
Use of statistical methods
Paper to CEMP 17/11/11 with case study examples of different assessment approaches
Single site example (DE)
Across a number of sites (UK)
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

6. CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty
Task 3.1B – Testing
Approach in line with ISO
Water quality – Sampling —Part 20: Guidance on the use of sampling data for decision making – Compliance with thresholds and classification systems.
Use specified percentile with an assigned confidence of failure.
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

7. CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty
Task 3.1B – Key issues
Compliance depends on sampling frequency
Probability of at least 1 sample failing
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

8. Task 3.1B – probability of failure
Probability a site exceeds an allowed time in failure for different sampling frequencies and observed fails
>50% confidence (face-value)
>95% confidence (significant)
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

9. Task 3.1B – Isoproturon example (DE)
Response
Additional intensive daily measurements showed short term pollution peaks occur over days and correlate with hydrological regime
 measures and source controls to reduce the short-term emissions
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

10. Task 3.1B – Example – Isoproturon (DE)
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

11. Task 3.1B – Example – 2009 MAC fails (UK)
LoQ issues
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

12. Task 3.1B – Statistical Methods – Why?
Bring consistency, recognising sampling regime
Identify the most significant failures
Defensible judgement for legal action
Target effort and resource to secure compliance
Level playing field
Fair comparison between places
They are not a means to ignore failure
All MAC exceedences should be investigated
To identify any appropriate pollution control measures
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty

13. Task 3.1B – Discussion points
Choice of percentile and confidence level depends on expert judgement
95 percentile (if monthly sampling) ?
50% confidence
identifies sites for further investigation into environmental risk (frequency, magnitude, duration of short-term pollution peaks)
Can identify need for broad scale source control ?
95% confidence – identify actions to secure compliance
Ceiling limit to respond to very high levels?
Should balance risk of false accusations (benefit of doubt) and risk of not recognising failure (fail-safe)
17-Nov-2011
CMEP Geel - Anja Duffek, Hannah Green, Markus Lehmann, John Batty