1. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl On the Role of Proficiency Testing Schemes in Laboratory Accreditation

2. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl 5.9Assuring the quality of test and calibration results The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following: a) regular use of certified reference materials and/or internal quality control using secondary reference materials; b) participation in interlaboratory comparison or proficiency-testing programmes; c) replicate tests or calibrations using the same or different methods; d) retesting or recalibration of retained items; e) correlation of results for different characteristics of an item. NOTE The selected methods should be appropriate for the type and volume of the work undertaken. ISO/IEC 17025 and PROFICIENCY TESTING

3. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl EXTERNAL QUALITY CONTROL: PROFICIENCY TESTING Inter-laboratory comparison Target value and standard deviation based upon results of participants Proficiency testing Organizer decides on target value and target standard deviation

4. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl The prime purpose is to allow participating laboratories to become aware of unsuspected errors in their work and to take remedial action. The participant can make three comparisons of performance: - with an externally determined standard of accuracy - with that of peer laboratories - with its own past performance PROFICIENCY TESTING, WHY:

5. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl PLEASE NOTE: Interlaboratory comparisons may be designed for purposes other than proficiency testing: - validation of methods - characterization of candidate reference materials - self-assessment of a laboratory in a performance test A precondition for selection is that the scheme is operated according to ISO/IEC Guide 43 and/or ILAC Guide G-13. ISO/IEC 17025 and PROFICIENCY TESTING

6. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING Why mandatory PT activities ? 1) ILAC requires a policy of the Accreditation Bodies related to the participation of accredited laboratories in PT. 2) ILAC makes recommendations 3) A National Accreditation Body may transfer this recommendation into its own policy, e.g.: …one PT activity prior to gaining accreditation and one activity relating to each major sub-area of major disciplines of a laboratory’s scope of accreditation at least every four years; where applicable and where practical….

7. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl Does any PT-scheme exist for the tests and samples normally analyzed by the laboratory ? Sources: - National accreditation bodies - Peer laboratories already participating - PT-organizers in the own country - Internet e.g. www.eptis.bam.de - http://www.ianz.govt.nz/aplac/documents/web_docs/APLACPT003.pdf - Accreditation & Quality Assurance journal ISO/IEC 17025 and PROFICIENCY TESTING

8. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING

9. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl PT-testing is not about ‘passing’ or ‘failing’ a test. PT-testing is about taking part and learning from the results One bad result does not make a laboratory bad. The challenge is to prevent that it repeats One good result does not make a laboratory supreme. The challenge is to repeat this performance Goal: consistent satisfactory performance ISO/IEC 17025 and PROFICIENCY TESTING

10. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl Auditors : KEEP IN MIND PT participation done under “daily conditions” ? Performance over sevral years with same PT organizer PT operated in accordance to ISO/IEC Guide 43-1 The scheme documentation and statistical protocol available ? ISO/IEC 17025 and PROFICIENCY TESTING

11. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl Does any PT-scheme exist for the tests and samples normally analyzed by the laboratory ? What IF NOT ? - Analysis of blind samples - Analysis of spiked samples - Analysis of multiple sub-samples - Exchange of samples with other laboratories - Use of more than one analytical method ISO/IEC 17025 and PROFICIENCY TESTING

12. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl QC/QA and method validation ?…. PT IS OFTEN A WRONG CHOICE Auditor: Check the method validation !! For assessment of sources of error ?….PT CAN BE A WRONG CHOICE Auditor: Check how and who did the PT measurement Because their clients say so ?….BUT WILL THEIR CLIENTS JUDGE ON BASIS OF 1 TEST ONLY ? Auditor: Check which client asked for this WRONG OBJECTIVES To get an impartial and independent view of the laboratory’s performance ?…. PT DOES NOT INDICATE THE REAL (DAY-BY-DAY) PERFORMANCE Auditor: Explain the PT report Because they have no suitable calibrants ?…. POSSIBLY, BUT CREATING A NEW RM BY PT IS A TIME-CONSUMING PROCESS… Auditor: What did they do so far ?

13. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl As double-blinded study ABCDEABCDE Aroclor 1242 062 % 0 0143 % benzo(k)fluoranthene 055 % 119 % 0 55 % chrysene 059 % 73 % 0 49 % arsenic84 %55 % 99 %57 % 61 % cadmium10 %12 % 98 %20 % 25 % lead27 %15 % 88 %16 % 34 % As PT test ABCDE Aroclor 1242120 %128 %102 %102 %111 % benzo(k)fluoranthene210 % 0 0 0210 % chrysene139 % 94 % 77 % 72 %158 % arsenic138 %106 % 91 %103 %112 % cadmium138 %100 % 89 %104 %102 % lead132 % 91 % 99 %112 %115 % Ref. R.Forman, Pittcon 2002 Proficiency testing : insight in the REAL performance ?

14. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl RELEVANT DOCUMENTATION Planned action List of PT’s in which is participated SOP’s for selection participation,administration, evaluation and follow-up ISO/IEC 17025 and PROFICIENCY TESTING

15. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl VALIDITY OF RESULTS OF PT The period of validity of the result obtained by a laboratory in a single round of a PT scheme is limited to the time that the laboratory performed the test !! If a laboratory achieves a satisfactory result in a single round, the result shall not be used to support a claim that the laboratory obtained reliable data on any other occasion !!! ISO/IEC 17025 and PROFICIENCY TESTING

16. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING

17. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING

18. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING

19. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING

20. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl Is the PT-scheme in which they participated relevant in view of the scope of accreditation ? Check: The normal IQC and results thereof: frequency, control charts, criteria, evaluation, remedial/corrective actions Type of material in daily measurement vs. IQC material vs. PT material Frequency of measuring routine samples of the type supplied as PT Measurand Measurand value in comparison with typical ‘daily’ range Deadline for reporting (turn-around time in comparison with routine work) Who analyzed it (senior staff member vs. technician ?) (Non) availability of IQC material -what else for routine control ? (Non) availability of suitable calibrants - how method validation ? Number of participants Expected number of different techniques and/or number of techniques, similar as applied in the laboratory Sample dimensions comparable to daily work conditions ? How may the majority have measured it: best measurement capability or routine? Frequency of this type of PT testing Results of previous PT’s on same material CHECKLIST FOR AUDITORS ISO/IEC 17025 and PROFICIENCY TESTING

21. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl ISO/IEC 17025 and PROFICIENCY TESTING DON’T FORGET: If the PT score is poor (|z| > 2 or |z|>3) : - Assess how the target uncertainty was determined; compare this with e.g. the associated value predicted by the Horwitz equation Sometime target uncertainties are unrealistic small, set by metrologists under best measurement conditions, not taking instead of “routine” conditions - Check your own measurement uncertainty

22. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl STUDY THE OVERALL PERFORMANCE OF ALL PARTICIPANTS: Unsatisfactory where the majority is satisfactory ? Unsatisfactory where a significant number performed poorly ? Both : indications of problems regarding methodology… ? then re-validate criteria for satisfactory performance incorrectly set ? then check performance of analysts…. ISO/IEC 17025 and PROFICIENCY TESTING

23. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl IS IT JUSTIFIABLE TO EXCLUDE OUTLIERS ? ISO-34 Clause 5.15.1 “…The reference material producer should never rely entirely on a statistical analysis of the characterization data when assessing the property values of interest. Outliers should not be excluded on purely statistical evidence until they have been thoroughly investigated and, where possible, the reasons for the discrepancies identified. Alternatively, the use of robust statistics may be appropriate in some cases….” ISO/IEC 17025 and PROFICIENCY TESTING

24. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl DO WE NEED STATISTICS IN PT ? Yes: - For “objective” demonstration of performance - For “objective” comparison of laboratories/ techniques/methods - If PT material has to be upgraded to RM No: - If participants use results for finding deficiencies - If fast turn-around times are necessary ISO/IEC 17025 and PROFICIENCY TESTING

25. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl Bad news: Many things in nature are not “normally” distributed. Good news: Much of what is not “normally” distributed in biology would be “normally” distributed if you took the logarithm of each data item. Bad news: Much of what is in books about statistics has to do with “normally” distributed data. Good news: Robust statistics provide useful information even if applied to not “normally” distributed data. ISO/IEC 17025 and PROFICIENCY TESTING

26. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl WHY DO ACCREDITED LABS PERFORM NOT BETTER THAN NON-ACCREDITED LABS? BECAUSE THESE COMPARISONS BETWEEN LAB-PERFORMANCES ARE BIASED !!! 1. Quality conscious laboratories join PT as part of their quality improvement strategy; other, less conscious laboratories are less likely to participate. 2. Non-accredited laboratories that perform badly may more likely to drop participation in PT than accredited labs, who need it to maintain their accreditation. So: there is an inclusion of only “good” non-accredited laboratories with both “good” and “bad” accredited laboratories !! 3. Accredited laboratories may even have no motivation to perform ‘super’ good; results are fit for the (clients) purpose 4. Accredited laboratories may have much more pressure on routine performance (large throughput) than non-accredited laboratories ISO/IEC 17025 and PROFICIENCY TESTING

27. Dr. Peter Bode, Associate Professor Delft University of Technology, Interfaculty Reactor Institute Mekelweg 15 2629JB Delft, The Netherlands Tel. 31-15-2783530 fax. 31-15-2783906 e-mail p.bode@iri.tudelft.nl Thank you for listening