Phil Smith Sales & Marketing Director Quality Management Ltd.

n Manchester – North of England n Established 1991 n 30 Employees n Nearly 20,000 samples per annum n 2400 Participating Laboratories n 53 Countries n 7 Separate PT Schemes

SUBJECT Proficiency Testing (External Quality Control) a Vital Part of Laboratory Quality Systems

OVERVIEW of PRESENTATION n Why a laboratory should participate n What is Proficiency testing n The essentials of a PT Scheme n How does it work n Accreditation of a PT Scheme n Relationship with Accreditation

WHAT IS PROFICIENCY TESTING? Proficiency Testing (PT) involves the regular distribution of stable homogenous test materials to participants for analysis for defined determinands. Participants results of analysis are then compared with those of the other participants. Proficiency Testing (PT) involves the regular distribution of stable homogenous test materials to participants for analysis for defined determinands. Participants results of analysis are then compared with those of the other participants. Proficiency Testing provides an independent means of testing and comparing individual laboratory/technician results of analysis with a peer group. Proficiency Testing provides an independent means of testing and comparing individual laboratory/technician results of analysis with a peer group.

USES OF PROFICIENCY TESTING Monitoring Performance Monitoring Performance Provides Peer Comparison Provides Peer Comparison Identifies Problems Identifies Problems Encourages Improvement Encourages Improvement Verification Verification Training Training Accreditation Accreditation Customer Demands Customer Demands

HOW DOES IT WORK

TYPICAL ROUND CYCLE Assigned Values FRI Results deadline FRI Report issued Final test date* Despatch date FRIWEDSTUES Week 5 Week 4 Week 3 Week 2 Week 1 * Recommended FaxPost Paper report Website

QMS Reports

ILAC* G13: 2000 Guidelines for the Requirements for the Competence of Providers of Proficiency Schemes *ILAC - International Laboratory Accreditation Co-operation THE ESSENTIALS OF A PT SCHEME ISO IEC Guide 43 – 1 Proficiency Testing by Interlaboratory Comparisons Part 1: Development and Operation of Proficiency Testing Schemes

ACCREDITATION OF PT PROVIDERS

KEY CRITERIA Test Materials must:- Test Materials must:- Contain the intended micro-organisms at the expected levels Contain the intended micro-organisms at the expected levels Be sufficiently Homogenous for purpose Be sufficiently Homogenous for purpose Be stable for at least the period of the PT round Be stable for at least the period of the PT round Statistics used to assess performance must:- Statistics used to assess performance must:- Be easy to understand and interpret Be easy to understand and interpret Enable direct comparison with other participants Enable direct comparison with other participants

STRAIN CHARACTERISTICS Source from recognised culture collections Source from recognised culture collections Method of Preservation must maintain characteristics Method of Preservation must maintain characteristics Verification after preservation Verification after preservation Appropriate to target sector e.g. analysis of food Appropriate to target sector e.g. analysis of food

STRAIN CHARACTERISTICS ATCC – American Type Culture Collection ATCC – American Type Culture Collection NCTC – National Collection of Type Cultures NCIMB – National Collection of Industrial & Marine Bacteria Use of Wild Strains - verification Use of Wild Strains - verification How QM assures the Authenticity of the strains used Lyophilised (Freeze dried) to maintain characteristics Lyophilised (Freeze dried) to maintain characteristics Morphological & biochemical tests after preservation Morphological & biochemical tests after preservation

STABILITY ILAC G13 – states that samples must be stable for at least the conduct of the round. e.g. Culture profile Staphylococcus aureus Gram +ve cocci Catalase and coagulase +ve Stability tests Mixed with Skimmed milk powder/Oatmeal and stored at 5, 25, 30,37 0 C Enumerate Check Purity Freeze-dry to produce initial stock (high number of micro-organisms in a powder format) Resuscitation on agar medium Storage on cryogenic beads Micro-organisms obtained from recognised culture collection Development tests before inclusion in QM Culture collection Development tests before inclusion in QM Culture collection

STABILITY

HOMOGENEITY Recognises that complete homogeneity cannot be achieved even in a liquid sample. Recognises that complete homogeneity cannot be achieved even in a liquid sample. Samples taken from batch at random and tested in duplicate for all target organisms. Samples taken from batch at random and tested in duplicate for all target organisms. International Harmonised Protocol for the Proficiency Testing of (chemical) Analytical laboratories states that the standard deviation (SD) for homogeneity results must be  0.3 times the acceptable range for participants. International Harmonised Protocol for the Proficiency Testing of (chemical) Analytical laboratories states that the standard deviation (SD) for homogeneity results must be  0.3 times the acceptable range for participants. e.g. If Target SD is 0.3 then homogeneity SD <0.09 ILAC G13 – states that the Provider shall be able to demonstrate that the test material is sufficiently homogenous for the particular proficiency testing scheme.

PRODUCTION OF TEST MATERIALS - SAMPLES Samples returned to QM and Stability tested End of Life Stability tests performed after 3 months storage Samples tested by participants Despatch Samples to participants One Set posted to return to QM Pre-despatch Quality control tests Homogeneity, Weight Production Samples are weighed (10g or 25g) Mix working stock with SMP or OM to produce bulk stock Check working stock Enumerate, Profile, Growth and biochemical reactions Dilute initial stock with irradiated SMP to produce working stock

ASSESSING PERFORMANCE Where: - X = mean = standard deviation or variation around the mean = standard deviation or variation around the mean % 3333 2222  3333 2222  X 68% 95% Diagram of a Normal Distribution

ASSESSING PERFORMANCE – Setting the Assigned Value Determining the Assigned Value n By Formulation – the assigned value is determined from knowledge of the test item formulation e.g preparing a solution of 1M hydrochloric acid. n Use of certified reference material (CRM) – the certified reference value will be used as the assigned value. n Use of reference material – a cheaper alternative to the use of certified reference materials, where a test-material is used that has been calibrated against a certified reference material. n Consensus values from expert laboratories – the expert laboratories should have demonstrable competence in the determination of the analyte(s) under tests, using validated methods. n Consensus values from participants – participant results are used to determine the assigned value, with consideration of the possible effects of extreme values.

A B C D E Assigned value Sub-samples containing the target micro-organism are distributed to at least 20 laboratories with instructions for testing ASSESSING PERFORMANCE – Setting the Assigned Value

Easy to understand and interpret Easy to understand and interpret Easily summarised in tabular or graphical form Easily summarised in tabular or graphical form Widely used and accepted Widely used and accepted Enable results to be compared over time Enable results to be compared over time Enable direct comparison with other participants Enable direct comparison with other participants ASSESSING PERFORMANCE – Performance Scoring

QM applies the Z-Score system Z = (x – X a ) QM applies the Z-Score system Z = (x – X a )  t  t Where: - x = the result reported by a participating laboratory Xa = the assigned value Xa = the assigned value t a target standard deviation  t = a target standard deviation When a data set conforms to a standardised normal distribution we can expect 95% of its values to lie within 2 SDs 99.7% of its values to lie within 3 SDs from the mean. When a data set conforms to a standardised normal distribution we can expect 95% of its values to lie within 2 SDs 99.7% of its values to lie within 3 SDs from the mean. Results with a Z-Score over 2 are therefore considered questionable because we would expect only 5% of correct measurements to be so different to the assigned value. Results with a Z-Score over 2 are therefore considered questionable because we would expect only 5% of correct measurements to be so different to the assigned value. Results with a Z-Score over 3 are considered unsatisfactory because we would expect only 0.3% of correct measurements to be so different to the assigned value. Results with a Z-Score over 3 are considered unsatisfactory because we would expect only 0.3% of correct measurements to be so different to the assigned value. ASSESSING PERFORMANCE – Performance Scoring

OTHER IMPORTANT CRITERIA How many laboratories are participating How many laboratories are participating Does the Scheme allow the user to choose their test regime Does the Scheme allow the user to choose their test regime Does the Provider have the required resources and finance Does the Provider have the required resources and finance How important is PT to the Provider How important is PT to the Provider Is the Scheme Accredited Is the Scheme Accredited

QM’s ACCREDITATION STATUS Based on ISO Guide 43-1:1997 and on the relevant elements of ISO 17025: 2000, applicable to the characterisation, homogeneity and stability testing of proficiency testing test materials. Additionally, relevant elements of ISO 9000:1994 are included to eliminate the need for separate recognition of compliance with ISO Accredited by UKAS (United Kingdom Accreditation Service) QM is accredited as a Proficiency Testing Provider Certificated to ISO

WHY A LABORATORY SHOULD PARTICIPATE Because you want to – pursuit of excellence Because you want to – pursuit of excellence Provides independent assessment of the quality of a laboratory’s performance in its routine work Provides independent assessment of the quality of a laboratory’s performance in its routine work Helps identifies potential problems Helps identifies potential problems Identifies competent laboratories for commercial purposes Identifies competent laboratories for commercial purposes Mandatory – Directives: - Mandatory – Directives: - 89/397/EEC Official Food Control Laboratories 89/397/EEC Official Food Control Laboratories 93/99/EEC Additional Measures for Food Control 93/99/EEC Additional Measures for Food Control Supports accreditation Supports accreditation Training, validating methods, internal Q C assessments Training, validating methods, internal Q C assessments

RELATIONSHIP WITH ACCREDITATION The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following. The laboratory shall have quality control procedures for monitoring the validity of tests and calibrations undertaken. The resulting data shall be recorded in such a way that trends are detectable and, where practicable, statistical techniques shall be applied to the reviewing of the results. This monitoring shall be planned and reviewed and may include, but not be limited to, the following. (a) Regular use of certified reference materials and/or internal quality control using secondary reference materials; (a) Regular use of certified reference materials and/or internal quality control using secondary reference materials; (b)Participation in interlaboratory comparisons or proficiency testing programmes; EN ISO/IEC EN ISO/IEC Assuring the quality of tests and calibration results 5.9 Assuring the quality of tests and calibration results

It contains all of the requirements that testing and calibration laboratory have to meet if they wish to demonstrate that they operate a quality system, are technically competent, and able to generate technically valid results. It contains all of the requirements that testing and calibration laboratory have to meet if they wish to demonstrate that they operate a quality system, are technically competent, and able to generate technically valid results. RELATIONSHIP WITH ACCREDITATION Introduction to EN ISO/IEC says: - Compliance with EN ISO/IEC sets the conditions that should prepare the laboratory to generate technically valid results Compliance with EN ISO/IEC does not measure the accuracy of the results produced by a laboratory Participation in a well run Proficiency Testing Scheme meets this need

QM CHEMISTRY PT SCHEMES Quality in Dairy Chemistry Quality in Meat Chemistry

OTHER QM MICRO PT SCHEMES Quality in Chocolate Quality in Beverages

QWAS - Quality in Water

CS – Calibration Standards Calibration Standards Analytical Quality Assurance Controls

Clean Water - Organics - Inorganics Waste Water- Organics - Inorganics Soils and Sediments Sludge and Sewage AQUACHECK Electronic results submission and reporting PT for Water and Environmental Chemistry