1. Guidance for water quality intercalibration
- draft -
PP2 – National Administration Romanian Waters
Chim. Carmen Hamchevici
Dr. biol. Gabriel Chiriac

2. Outline Terms and definitions needed Objective Methods Results
General (widely used)
Specific (adjusted to needs)
Results
Interpretation of results
Conclusions

3. Terms and definitions Intercalibration
Strictly metrological: (
to determine, check, or rectify the graduation of any instrument giving quantitative measurements
Extended
A state achieved by a group of laboratories engaged in a monitoring program in which they produce and maintain compatible data outputs (McGraw-Hill Dictionary of Scientific & Technical Terms, 2003)
Current use at the EU level given by the WFD Guidance Documents:
the intercalibration process is aimed at consistency and comparability of the classification results of the monitoring systems operated by each Member State for the biological quality elements (CIS Guidance Document no. 14 – Guidance on the intercalibration Process )

4. Objective intercalibration
The process, procedures and activities used to ensure that the several laboratories engaged in a monitoring program can produce compatible data
compatible data outputs are achieved and this situation is maintained
the laboratories can be said to be intercalibrated
active process between laboratories that includes all steps from sampling to analyses
achieving the same accurate results regardless of the method or lab
Taylor, 1987

5. Achieving the objective (already done?!)
Quality Assurance
Quality System in place
(accreditation acc to ISO/IEC 17025:2005)
Technical competence
Objective (partially) achieved

6. Methods (1)
General
Proficiency testing (PT): the evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons (ILC)
Interlaboratory comparisons (ILC): organization, performance and evaluation of measurements or tests on the same or similar items by two or more laboratories […] in accordance with predetermined conditions
Other purposes of ILC than PT
to evaluate the performance characteristics of a method
to characterise a reference material
to compare results of two or more laboratories on their own initiative
to support statements of the equivalence of measurement of National Metrology Institutes
Source: International Laboratory Accreditation Cooperation (ILAC) Policy for Participation in PT Activities (ILAC-P9:06/2014)

7. Proficiency Testing schemes (1)
evaluation of the performance of laboratories and monitor laboratories’ continuing performance
identification of problems in laboratories and initiation of corrective actions
establishment of the effectiveness and comparability of test or measurement methods
provision of additional confidence to laboratory customers
identification of inter-laboratory differences

8. Provider Participant Proficiency Testing schemes – who is involved?
organization which takes responsibility for all tasks in the development and operation of a proficiency testing scheme
Provider
laboratory, organization or individual that receives proficiency test items and submits results for review by the proficiency testing provider
Participant

9. Proficiency Testing schemes – use of analytical methods
the laboratory should use its routine method
the choice might be limited by e.g. legal regulations
specified in the reporting sheet
provider has a policy and follows a procedure regarding comparison of results obtained by different methods

10. Proficiency Testing schemes – assigned value
one of the most critical features of a PT
inappropriate value will drastically reduce the value of the scheme
of the ISO/IEC 17043:2010: the PT provider shall document the procedure for determining the assigned values for the measurands or characteristics in a particular PT scheme

11. Assigned value (1)
Reference Certified Material: ideal test material for a PT
high costs
limited availability
Consensus of “Expert Laboratories“: mean of analysis by expert laboratories
difficult to ensure the accuracy of the reference measurements
„nobody is perfect“: there might be doubts among the participants if the result of the expert laboratories deviates from the mean of the participants

12. Assigned value (2) Formulated or “Synthetic” test materials
materials spiked with the analyte to a known extent
if material does not contain significant amounts of the analyte,
assigned value directly from added amount
difficult to achieve sufficient homogeneity, especially with solid materials
analyte might be bound in a different chemical form
Consensus of participants
easiest and cheapest, widely used
if method for analysis is easy - good estimate of „true“ value
consensus value might be seriously biased
no consensus at all
these circumstances are frequent in trace analysis

13. Methods for calculating consensus value
Arithmetic mean: requires an outlier test that assumes normal distribution (normally not true in trace analysis)
Median value:
not affected by outlying data
makes not full use of the information content of the data
Robust mean:
„trimmed“ data; a certain part of the data on both tails of the data set is excluded prior to the calculation of the mean
e.g. mean of interquartile range (mean of data between the first and the third quartile of the data set)
Huber statistics
Source: Koch, M.: Interlaboratory tests in Wenclawiak, Koch, Hadjicostas (eds.), 2003 Quality Assurance in Analytical Chemistry – Training and Teaching

14. Assessment of performance
Basics: assigned value and standard deviation for proficiency assessment
Means: z-score
Z – score = (𝒙−𝝁) 𝝈
X – result reported by participant
𝝁 – assigned value
𝝈 – standard deviation for proficiency assessment
Source: Koch, M.: Interlaboratory tests in Wenclawiak, Koch, Hadjicostas (eds.), 2003 Quality Assurance in Analytical Chemistry – Training and Teaching

15. Performance criteria
│z│ ≤ 2,0 the score indicates “satisfactory” performance and generates no signal
2.0 < │z│ < 3.0 the score indicates “questionable” performance and generates a warning signal
│z│ ≥ 3.0 the score indicates “unsatisfactory” performance and generates an action signal

16. Example of z-score diagram
Source: Koch, M.: Interlaboratory tests in Wenclawiak, Koch, Hadjicostas (eds.), 2003 Quality Assurance in Analytical Chemistry – Training and Teaching

17. References used in PTs
ISO/CEI 17043:2010 Conformity assessment – General requirements for proficiency testing
ILAC-P9:06/2014: International Laboratory Accreditation Cooperation Policy for Participation in PT Activities
ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories
EURACHEM: Selection, use and interpretation of Proficiency Testing (PT) schemes, Second Edition 2011
ISO 13528: 2005 Statistical methods for use in proficiency testing by inter-laboratory comparisons

18. Methods (2)
Specific
Transboundary QA/QC schemes: QUALCO-DANUBE the danubian laboratories reporting to the Trans-National Monitoring Network of the ICPDR
Bilateral Agreements between neighboring countries
Common or parallel actions during different types of activities: research projects, investigative monitoring (i.e. Joint Danube Surveys)

19. PT scheme Qualco-Danube (1)
established and implemented as the primary inter-laboratory quality control program in the Danube basin
started in 1993, extended in 1995 to 11 National Reference Labs (NRLs) and from 1996 to 19 NRLs
Provider: (until 2012) the Institute for Water Pollution Control of VITUKI, Budapest, Hungary
present WESSLING (HU)
Quarterly distribution every year (until 2012)
2013 – present: flexible (nutrients / every year, heavy metals / every 2 years, organics / every 3 tears)
Sample preparation and evaluation scheme for AQC in the Danube river basin (Five-years Report on Water Quality in Danube River Basin Based on TransNational Montoring Network, 2003)

20. PT scheme Qualco-Danube - Assessment of performance
based on the tolerance intervals (until 2008)
Z-score (2008-present)

21. ILC through Bilateral Agreement
General framework: set within Protocol
Common sampling campaigns
Frequency and parameters: based on the transboundary programmes requirements
Analytical methods: routine
Analysis of compliance of minimum technical requirements from QA/QC Directive (2009/90/EC)
Bilateral meetings for discussions on results and agreement
Analysis of results from other PT schemes of interest
Investigation of unsatisfactory or questionable PT results
Monitoring of PT performance over time

22. ILC through common or parallel actions
Sampling during different types of activities: research projects, investigative monitoring (i.e. Joint Danube Surveys)
Objective: harmonisation of sampling techniques, analysis methods and assessment method
Technical Reports available: supporting documents for further analysis

23. Conclusions Laboratory intercalibration is not an easy task, but...
PT schemes are
Required by the accreditation process
Available and fit for purpose
Widely used
Statement of laboratory’s performance
Transboundary PT schemes – operational and good enough
Bilateral Agreement in place – on regular basis and targeted to specific needs

24. Thank you for your attention!
Contacts
Thank you for your attention!
Investing in your future!
Romania-Bulgaria Cross Border Cooperation Programme is co-financed by the European Union through the European Regional Development Fund