What’s the best quality program for your lab? Grant Maxie, DVM, PhD, DACVP Director, Animal Health Laboratory; Co-Executive Director Nadine Ryan, BSc Quality Manager, Laboratory Services Division, University of Guelph CAHLN-RCTLSA annual meeting, Calgary, AB June 8, 2010

What’s the best quality program for your lab? Why have a quality program? Alternatives Comparisons Components of a quality system Test validation Discussion/Questions 2 2

Why have a quality program? meet the expectations of clients especially if a requirement of contracts ensure credibility of results – locally, nationally, and internationally deal with complaints and solve problems root cause analysis minimize re-work employee recruitment and retention business sustainability and continuity continuous improvement!! 3

Stakeholders/partners of the AHL: Livestock producers & animal owners Provincial agriculture ministry Practitioners CFIA Industry AHL WOAH (OIE) Public health University 4 4 4

AHL function and structure Client service Laboratory testing Quality program 5 5 5

What client services do AHL’s provide? timely, accurate diagnostic testing credible consultative services support for teaching and research activities strongly client-focused data for risk management - wellness, on-farm food safety, quality assurance animal health surveillance data - public health link (CAHSN – CNPHI) 6

Quality system support of client service Vision Mission statement Values Quality goals and objectives Quality manual – procedures Requires: Periodic strategic planning Management and staff involvement and commitment 7

Overarching quality principles say what you do do what you say prove it improve upon it 8

Accreditation routes ISO 9001:2008 Quality Management Systems registration, customer-focused ISO/IEC 17025 General Requirements for the Competence of Testing and Calibration Laboratories test-specific scope accreditation AAVLD Requirements for an Accredited Veterinary Medical Diagnostic Laboratory, v 4.3 incorporates ISO/IEC 17025/OIE standards OECD Good Laboratory Practice (GLP) 9

1. ISO 9001:2008 registration focuses on customer satisfaction and managing the organization’s interconnected processes to meet the customer’s and regulatory requirements includes management responsibility, resource management, product (service) realization, and measurement, analysis and improvement the requirements of ISO 9001 are generic and apply to all organizations involves annual surveillance audits and a registration audit every 3 years general in scope, and does not involve assessment of competence to perform individual tests Cost = ~$5,000 + expenses / year 10

Why ISO 9001 for a lab? Continuous improvement cycle – plan, do, check, act For a laboratory – substitute “service” for “product” 11

2. ISO/IEC 17025 accreditation expands upon the ISO 9001 generic requirements, rewriting to specific laboratory requirements and adding technical requirements relevant to the laboratory’s competence to produce accurate and reliable results federal labs (CFIA, USDA) typically operate under an ISO/IEC 17025 standard, and work subcontracted from them must meet this standard biennial external audits are required, questionnaire in intervening years accreditation is scope-specific (limited to audited tests) and posted on the website of the national accrediting body, e.g., Standards Council of Canada (SCC) Cost = ~ $6,000 - $29,000 + expenses / year depending on # of tests 12

Why ISO/IEC 17025 for vet dx lab? Internationally recognized Global acceptance of laboratory data Canada’s accreditation bodies (SCC & CALA) are members of the International Laboratory Accreditation Cooperation (ILAC) and evaluated by other members The ILAC network consists of 135 bodies representing 88 different economies 13

3. AAVLD accreditation publically supported veterinary diagnostic labs only has evolved from a peer-help process now based on the OIE Quality Standard and Guidelines for Veterinary Laboratories: Infectious Diseases, 2008, which is consistent with the ISO/IEC 17025 standard external audits vary, with a maximum of 5 years between audits 42 accredited labs in North America – BC and ON in Canada annual fee of $500, audit year = $1,000 + $500/branch lab + local expenses 14

- - Accreditation files AAVLD website - Member side - - Accreditation files http://data.memberclicks.com/site/aavld/Comparison_of_AAVLD_-_ISO_17025_Accreditation_Programs_AC_102_4_3_v_1___2_.pdf 15

4. Other accreditation options? OECD GLP additional quality documentation required contract research lab (CRO) specialty ISO/IEC 17025 flexible scope (tentative) include multiple test/test types within a broader scope, e.g., “veterinary microbiology” USA - A2LA (American Association for Laboratory Accreditation) http://www.a2la.org/ Veterinary Laboratory Accreditation Program

Components of a quality program ISO section 4 – Management requirements 4.1 organization and management – authorities 4.2 quality system – quality policy and objectives documented in a quality manual 4.3 document control 4.4 review of requests, tenders, contracts 4.5 subcontracting of test services 4.6 purchasing of services and supplies 4.7 complaints 4.8 control of nonconforming testing and test results 4.9 corrective and preventive actions 4.10 records management 4.11 internal audits 4.12 periodic management reviews 17

Components of a quality program ISO section 5 – Technical requirements 5.1 general 5.2 personnel (competence, training) 5.3 accommodation and environmental conditions 5.4 test methods and method validation SOPs, documented proficiency 5.5 equipment – maintenance, calibration 5.6 measurement traceability – reference materials 5.7 specimens - sampling 5.8 handling of specimens 5.9 ensuring the quality of test results - QC 5.10 reporting of test results 18

Challenges in maintaining any Q program staying current – annual/biennial review/ updating of all materials internal auditing – how frequent, followup on CAPAs trend analysis – QC on tests, trends in nonconformances staff commitment – cost-benefit analysis measurable objectives

Why validate tests? implications for patients (individuals or groups), need to support evidence-based medicine within lab confidence in test accuracy & precision understand biases (false positives, negatives) competent client service DVMs are more sophisticated contract/industry requirement maintain lab revenue! 20

What is a valid test? an assay that reliably produces correct results e.g., correctly identifies animals as positive or negative for an analyte, antibody, nucleic acid, antigen, etc. 21

Challenges to validation information is sometimes available publications package inserts pressure to release new tests resources, i.e., $$ & time logistical issues availability of samples from target & control populations numbers collection 22

AHL test validation process feasibility diagnostician determines the need for a new test new technologies (molecular biology) industry needs economic ‘old’ tests are not included discuss amongst selected users in lab 23

Diagnostic validation considerations how will the test be used target population & reference population sample size a priori estimate of Se, Sp statistical confidence sampling random, convenience, etc. inclusion/exclusion bias reduction (blind evaluation) gold standard 24

SOP analytical validation 25

SOP diagnostic (field) validation 26

References & resources on how to validate tests OIE Manual of Standards traditional http://www.oie.int/norms/mmanual/A_00012.html PCR http://www.oie.int/norms/mmanual/A_00014.html Veterinary Epidemiology Textbooks Preventive Veterinary Medicine, 2000, Volume 45, Special Issue: Validation and application of diagnostic tests used in veterinary epidemiologic studies AAVLD Workshop on Test Validation AAVLD Epidemiology Committee 27

Summary The bottom line of an effective and auditable laboratory quality program is client satisfaction through rigorous monitoring of valid tests and continuous improvement of processes. 28

29 29