Guidance for water quality intercalibration - draft - PP2 – National Administration Romanian Waters Chim. Carmen Hamchevici Dr. biol. Gabriel Chiriac

Outline Terms and definitions needed Objective Methods Results General (widely used) Specific (adjusted to needs) Results Interpretation of results Conclusions

Terms and definitions Intercalibration Strictly metrological: (http://dictionary.reference.com/browse/intercalibration): to determine, check, or rectify the graduation of any instrument giving quantitative measurements Extended A state achieved by a group of laboratories engaged in a monitoring program in which they produce and maintain compatible data outputs (McGraw-Hill Dictionary of Scientific & Technical Terms, 2003) Current use at the EU level given by the WFD Guidance Documents: the intercalibration process is aimed at consistency and comparability of the classification results of the monitoring systems operated by each Member State for the biological quality elements (CIS Guidance Document no. 14 – Guidance on the intercalibration Process 2004-2006)

Objective intercalibration The process, procedures and activities used to ensure that the several laboratories engaged in a monitoring program can produce compatible data compatible data outputs are achieved and this situation is maintained the laboratories can be said to be intercalibrated active process between laboratories that includes all steps from sampling to analyses achieving the same accurate results regardless of the method or lab Taylor, 1987

Achieving the objective (already done?!) Quality Assurance Quality System in place (accreditation acc to ISO/IEC 17025:2005) Technical competence Objective (partially) achieved

Methods (1) General Proficiency testing (PT): the evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons (ILC) Interlaboratory comparisons (ILC): organization, performance and evaluation of measurements or tests on the same or similar items by two or more laboratories […] in accordance with predetermined conditions Other purposes of ILC than PT to evaluate the performance characteristics of a method to characterise a reference material to compare results of two or more laboratories on their own initiative to support statements of the equivalence of measurement of National Metrology Institutes Source: International Laboratory Accreditation Cooperation (ILAC) Policy for Participation in PT Activities (ILAC-P9:06/2014)

Proficiency Testing schemes (1) evaluation of the performance of laboratories and monitor laboratories’ continuing performance identification of problems in laboratories and initiation of corrective actions establishment of the effectiveness and comparability of test or measurement methods provision of additional confidence to laboratory customers identification of inter-laboratory differences

Provider Participant Proficiency Testing schemes – who is involved? organization which takes responsibility for all tasks in the development and operation of a proficiency testing scheme Provider laboratory, organization or individual that receives proficiency test items and submits results for review by the proficiency testing provider Participant

Proficiency Testing schemes – use of analytical methods the laboratory should use its routine method the choice might be limited by e.g. legal regulations specified in the reporting sheet provider has a policy and follows a procedure regarding comparison of results obtained by different methods

Proficiency Testing schemes – assigned value one of the most critical features of a PT inappropriate value will drastically reduce the value of the scheme 4.4.5.1 of the ISO/IEC 17043:2010: the PT provider shall document the procedure for determining the assigned values for the measurands or characteristics in a particular PT scheme

Assigned value (1) Reference Certified Material: ideal test material for a PT high costs limited availability Consensus of “Expert Laboratories“: mean of analysis by expert laboratories difficult to ensure the accuracy of the reference measurements „nobody is perfect“: there might be doubts among the participants if the result of the expert laboratories deviates from the mean of the participants

Assigned value (2) Formulated or “Synthetic” test materials materials spiked with the analyte to a known extent if material does not contain significant amounts of the analyte, assigned value directly from added amount difficult to achieve sufficient homogeneity, especially with solid materials analyte might be bound in a different chemical form Consensus of participants easiest and cheapest, widely used if method for analysis is easy - good estimate of „true“ value consensus value might be seriously biased no consensus at all these circumstances are frequent in trace analysis

Methods for calculating consensus value Arithmetic mean: requires an outlier test that assumes normal distribution (normally not true in trace analysis) Median value: not affected by outlying data makes not full use of the information content of the data Robust mean: „trimmed“ data; a certain part of the data on both tails of the data set is excluded prior to the calculation of the mean e.g. mean of interquartile range (mean of data between the first and the third quartile of the data set) Huber statistics Source: Koch, M.: Interlaboratory tests in Wenclawiak, Koch, Hadjicostas (eds.), 2003 Quality Assurance in Analytical Chemistry – Training and Teaching

Assessment of performance Basics: assigned value and standard deviation for proficiency assessment Means: z-score Z – score = (𝒙−𝝁) 𝝈 X – result reported by participant 𝝁 – assigned value 𝝈 – standard deviation for proficiency assessment Source: Koch, M.: Interlaboratory tests in Wenclawiak, Koch, Hadjicostas (eds.), 2003 Quality Assurance in Analytical Chemistry – Training and Teaching

Performance criteria │z│ ≤ 2,0 the score indicates “satisfactory” performance and generates no signal 2.0 < │z│ < 3.0 the score indicates “questionable” performance and generates a warning signal │z│ ≥ 3.0 the score indicates “unsatisfactory” performance and generates an action signal

Example of z-score diagram Source: Koch, M.: Interlaboratory tests in Wenclawiak, Koch, Hadjicostas (eds.), 2003 Quality Assurance in Analytical Chemistry – Training and Teaching

References used in PTs ISO/CEI 17043:2010 Conformity assessment – General requirements for proficiency testing ILAC-P9:06/2014: International Laboratory Accreditation Cooperation Policy for Participation in PT Activities ISO/IEC 17025:2005 General requirements for the competence of testing and calibration laboratories EURACHEM: Selection, use and interpretation of Proficiency Testing (PT) schemes, Second Edition 2011 ISO 13528: 2005 Statistical methods for use in proficiency testing by inter-laboratory comparisons

Methods (2) Specific Transboundary QA/QC schemes: QUALCO-DANUBE the danubian laboratories reporting to the Trans-National Monitoring Network of the ICPDR Bilateral Agreements between neighboring countries Common or parallel actions during different types of activities: research projects, investigative monitoring (i.e. Joint Danube Surveys)

PT scheme Qualco-Danube (1) established and implemented as the primary inter-laboratory quality control program in the Danube basin started in 1993, extended in 1995 to 11 National Reference Labs (NRLs) and from 1996 to 19 NRLs Provider: (until 2012) the Institute for Water Pollution Control of VITUKI, Budapest, Hungary 2013 - present WESSLING (HU) Quarterly distribution every year (until 2012) 2013 – present: flexible (nutrients / every year, heavy metals / every 2 years, organics / every 3 tears) Sample preparation and evaluation scheme for AQC in the Danube river basin (Five-years Report on Water Quality in Danube River Basin Based on TransNational Montoring Network, 2003)

PT scheme Qualco-Danube - Assessment of performance based on the tolerance intervals (until 2008) Z-score (2008-present)

ILC through Bilateral Agreement General framework: set within Protocol Common sampling campaigns Frequency and parameters: based on the transboundary programmes requirements Analytical methods: routine Analysis of compliance of minimum technical requirements from QA/QC Directive (2009/90/EC) Bilateral meetings for discussions on results and agreement Analysis of results from other PT schemes of interest Investigation of unsatisfactory or questionable PT results Monitoring of PT performance over time

ILC through common or parallel actions Sampling during different types of activities: research projects, investigative monitoring (i.e. Joint Danube Surveys) Objective: harmonisation of sampling techniques, analysis methods and assessment method Technical Reports available: supporting documents for further analysis

Conclusions Laboratory intercalibration is not an easy task, but... PT schemes are Required by the accreditation process Available and fit for purpose Widely used Statement of laboratory’s performance Transboundary PT schemes – operational and good enough Bilateral Agreement in place – on regular basis and targeted to specific needs

Thank you for your attention! Contacts Thank you for your attention! www.cbcromaniabulgaria.eu Investing in your future! Romania-Bulgaria Cross Border Cooperation Programme 2007-2013 is co-financed by the European Union through the European Regional Development Fund