1. Roger Brauninger American Association for Laboratory Accreditation Frederick, Maryland February 3, 2016 Government Food and Feed Accreditation meeting Ensuring a Smooth ISO/IEC 17025 Accreditation Assessment Tips on Reducing the Uncertainty and Anxiety of an Upcoming Auditor Visit.

2. Topics Pretest Exercise What is Accreditation? Preparing for assessment Requirements, records & documentation Gap & Preassessment audits On-site assessment actions Potential “Rabbit Holes

3. Pretest Exercise

4. Quizz All of the questions in this quiz relate to requirements of the current edition of ISO/IEC 17025. Each question offers you four multiple-choice answers ( a, b, c, or d ). When ready hold up the colored card(s) which correspond to the correct answer(s).

5. 1.In what year was the current edition of ISO/IEC 17025 published? a. 1982 b. 1999 c. 2005 d. 1990

6. 1.In what year was the current edition of ISO/IEC 17025 published? a. 1982 b. 1999 c. 2005 d. 1990

7. 2.ISO/IEC 17025 requires the laboratory to conduct audits of its activities______: a. Every 6 months b. At least once/year c. At predetermined intervals decided by the laboratory d. Whenever testing problems are found

8. 2.ISO/IEC 17025 requires the laboratory to conduct audits of its activities______: a. Every 6 months b. At least once/year c. At predetermined intervals decided by the laboratory d. Whenever testing problems are found

9. 3.ISO/IEC 17025 requires that management reviews are conducted_______: a. Every 6 months b. At least once/year c. At predetermined intervals decided by the laboratory d. Whenever testing problems are found

10. 3.ISO/IEC 17025 requires that management reviews are conducted_______: a. Every 6 months b. At least once/year c. At predetermined intervals decided by the laboratory d. Whenever testing problems are found

11. 4.ISO/IEC 17025 requires the laboratory to notify the client in writing if_______: a. Any of the tests are to be subcontracted b. The test item is unsuitable for testing c. Audits reveal doubts about the test results d. A non-standard test method is to be used

12. 4.ISO/IEC 17025 requires the laboratory to notify the client in writing if_______: a. Any of the tests are to be subcontracted b. The test item is unsuitable for testing c. Audits reveal doubts about the test results d. A non-standard test method is to be used

13. 5. ISO/IEC 17025 does not require a laboratory to document________: a. Instructions for the operation of testing and measuring equipment b. Its procedures for purchasing and storing laboratory consumables c. Its procedures for subcontracting testing work d. Its procedures for document control

14. 5. ISO/IEC 17025 does not require a laboratory to document________: a. Instructions for the operation of testing and measuring equipment b. Its procedures for purchasing and storing laboratory consumables c. Its procedures for subcontracting testing work d. Its procedures for document control

15. 6.ISO/IEC 17025 requires a laboratory to keep records of_______: a. Customer complaints b. Staff training c. Results of audits d. Management reviews

16. 6.ISO/IEC 17025 requires a laboratory to keep records of_______: a. Customer complaints b. Staff training c. Results of audits d. Management reviews

17. 7.ISO/IEC 17025 requires the laboratory to ________: a. Control access to testing and measurement areas b. Protect confidentiality of its test results c. Comply with laws relating to laboratory safety d. Have air conditioning which controls temperature and humidity

18. 7.ISO/IEC 17025 requires the laboratory to ________: a. Control access to testing and measurement areas b. Protect confidentiality of its test results c. Comply with laws relating to laboratory safety d. Have air conditioning which controls temperature and humidity

19. 8.With respect to test equipment, a lab that complies with ISO/IEC 17025 must________: a. Ensure that all such equipment which affects accuracy must be calibrated b. Own and control all of the equipment that it uses c. Attach a label to the equipment showing its calibration status d. Stop using the equipment if it is not operating correctly

20. 8.With respect to test equipment, a lab that complies with ISO/IEC 17025 must________: a. Ensure that all such equipment which affects accuracy must be calibrated b. Own and control all of the equipment that it uses c. Attach a label to the equipment showing its calibration status d. Stop using the equipment if it is not operating correctly

21. 9.Before subcontracting any testing work to another laboratory, a lab which complies with ISO/IEC 17025 must first _______: a. Obtain a copy of the other laboratory’s quality manual b. Ensure that the other laboratory is accredited to ISO/IEC 17025 c. Ensure that the other laboratory is competent d. Obtain a copy of the other laboratory’s price list

22. 9.Before subcontracting any testing work to another laboratory, a lab which complies with ISO/IEC 17025 must first _______: a. Obtain a copy of the other laboratory’s quality manual b. Ensure that the other laboratory is accredited to ISO/IEC 17025 c. Ensure that the other laboratory is competent d. Obtain a copy of the other laboratory’s price list

23. 10. A laboratory that complies with ISO/IEC 17025 would also meet the principles of______? a. ISO 9001-2000 Quality Management Systems b. EN45001-1989 General criteria for theoperation of testing laboratories c. ANSI/ASQC Q.2-1991 Guidelines for Quality Management and Systems d. ISO/IEC 17011 General Requirements for Laboratory Accreditation

24. 10. A laboratory that complies with ISO/IEC 17025 would also meet the principles of______? a. ISO 9001-2000 Quality Management Systems b. EN45001-1989 General criteria for the operation of testing laboratories c. ANSI/ASQC Q.2-1991 Guidelines for Quality Management and Systems d. ISO/IEC 17011 General Requirements for Laboratory Accreditation

25. WHAT IS ACCREDITATION?

26. What is Accreditation? Process, to determine an organization’s competence to carry out specific tasks : By independent 3 rd party accreditation body Recognizes via a certificate and scope Requires periodic monitoring of performance and regular reassessment Prerequisites: compliance with ISO/IEC 17025.

27. Accreditation Ten Second Tutorial Do say Doing what you say you are doing prove And being able to prove it!

28. Three Critical Thoughts “say” Does the laboratory “say” what they do ? Is there written documents (policies, procedures, arrangements) that meet the requirements of ISO 17025? “ do ” Does the laboratory “ do ” what they say? Are they in compliance with their own quality system, test methods and ISO 17025? “ prove ” And can they “ prove ” it with their records? Ranging from having training records to standards preparation to work books to client reports to audit reports and everything in between?

29. Scope of Accreditation Although ISO 17025 is written as though all the methods used by a laboratory are covered by the standard, this is rarely the case. Methods which are performed infrequently are difficult to accredit Hard to demonstrate a track record of performance. In such cases a large level of quality control will be required by the assessors. Methods with little objective content are difficult to be able to get accredited Consistency in application cannot be guaranteed

30. Quality Manual Lays out Quality System roadmap No set format required Level of detail based on structure/ function of the laboratory and its’ needs Laboratory’s policies must be included or referenced Often follows 17025 sequence

31. AOAC Guidelines Started in 1998 with FLAWG precursor. Additional criteria for: Personnel Equipment cal and verification Reagents Proficiency testing Internal QC frequency

32. Areas Needing Early Start... Proficiency Testing data needed for: Each major matrix of concern For as many analytes/methods as possible Estimates of Measurement Uncertainty Develop Procedures, ID contributing factors Training on calculation (e.g. LCS data) Traceability of Calibrations, if in-house… Through an unbroken chain of comparisons, with each step having stated uncertainties Demonstrable competence performing calibration

33. Preparing … Study to understand requirements Review/ update all technical and quality procedures, QM policies with technical staff Get involvement from the technical staff in ensuring the lab’s compliance to its QMS Evaluate readiness with internal audits/ management review or external help

34. Internal audits vs. management reviews These are two distinct activities: Internal audits verify conformance to the documented system and confirms the management system is in compliance with the standard Management reviews determine if the management system policies and procedures are suitable and effective in generating quality data, meet the objectives of the laboratory and if improvements to meet changing needs are required

35. Gap Assessment (1 st or 3 rd party) Prospective focus sets direction and planning. Looks at A-Z / notes the degree of compliance and Identifies corresponding actions to be taken. Internal audit (1 st or 3 rd party) Retrospective focus on compliance/gathering evidence. Includes QMS, technical competence and AB’s policies. Multidimensional, looking at implementation. Accreditation body Pre-assessment (3 rd party) Mix of both: prospective focus and on compliance. Primary focus is not on technical. Types of Preparation Audits

36. During the assessment the assessor determines that there are major non-compliances with the criteria that would lead to a large number of significant deficiencies. In this case, the assessor identifies the potential deficiencies and suggests to the laboratory that the full formal assessment should wait until the problems have been addressed. This first identification of the problems would be considered a pre- assessment and is optional (i.e., lab could decide to go ahead anyway). Pre-assessment: Unplanned for scenario

37. During prior document review the assessor identifies major gaps in QM/ lab documentation. The applicant is given time to correct and full assessment follows later. In this case, the assessor identifies the potential deficiencies and suggests to the laboratory that the full formal assessment should wait until the problems have been addressed. Pre-assessment: Planned Triggering scenario

38. At time of application lab requests a pre-assessment because it is unsure of its documentation or system. In this case, the assessor identifies the potential deficiencies and full formal assessment takes place after the problems have been addressed. Ideal time is after both a gap assessment and Internal audit. Pre-assessment: Planned Triggering scenario

39. Management System ISO 17025 requirements explained, Reviews the status of the labs’ preparations, Responds to suggestions on bringing deficiencies into compliance. Technical General technical capability evaluated, Accommodations, Environment, Log books Pre-assessment: Report

40. INTERPRETING SOME “ISO” TERMS

41. Definitions Shall = imperative Should = strong recommendation Policy = rules – the “what” Procedure = step-by-step – the “how” Quality System = written policies, procedures, arrangements, etc. (all of the documents)

42. “Records” Record List Register Formulate Validate Evaluate In Writing Monitor Feedback ISO 17025 defines these words as “records” and specifies needed evidence of how actions were done e.g. (the “proof”):

43. “Documents” Document Policy Procedure Instruction System Plan Define Specify Pre-defined Pre-determined Arrangement ISO 17025 considers these words as those that describe how activities are to be done (the “what” and “how”):

44. “Weasel Words” “Where practicable” “Where necessary” “Including but not limited to” “Where appropriate” “Where possible” “Whenever reasonable” “If relevant” “Where relevant”

45. Shall “plus” Shall + “include” –whatever is listed after must be in the procedure or record. (unless it is accompanied by a weasel) Shall + “ensure” - The objective evidence (outcome) must show that the stated goal listed in the standard has been met.

46. ISO/IEC 17025 Implementations Implementations Records Records Documentation Documentation

47. 4.1 - Organization Green - 17025 Dk. Blue - 17025/Policies & Procedures Lt. Blue - AOAC

48. 4.2 - Quality System Green - 17025 Dk. Blue - 17025/Policies & Procedures Lt. Blue - AOAC

49. 4.3 - Document Control Green - 17025 Dk. Blue - 17025/Policies & Procedures

50. 4.4 - Review of Requests, Contracts, & Tenders Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records

51. 4.5 - Subcontracting Green - 17025 White - 17025/Records

52. 4.6 - Purchasing Services & Supplies Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

53. 4.6 - Purchasing Services & Supplies (cont.) Green - 17025 Lt. Blue - AOAC

54. 4.7 - Service to Client Green - 17025

55. 4.8 - Complaints Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records

56. 4.9 - Control of Non-Conforming Work Green - 17025 Dk. Blue - 17025/Policies & Procedures

57. 4.11 - Corrective Action Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records

58. 4.12 - Preventative Action Green - 17025 Dk. Blue - 17025/Policies & Procedures

59. 4.13 - Records Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

60. 4.14 - Internal Audits Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records

61. 4.15 - Management Review Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

62. 5.2 - Personnel Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

63. 5.3 - Accommodation & Environmental Conditions Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

64. 5.4 - Test and Calibration Methods Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records

65. 5.5 - Equipment Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

66. 5.6 - Measurement Traceability Green - 17025 Dk. Blue - 17025/Policies & Procedures Lt. Blue - AOAC

67. 5.7 - Sampling Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

68. 5.8 - Handling of Test Items Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

69. 5.9 - Assuring Quality of Test Results Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records Lt. Blue - AOAC

70. Additional PT Requirements Green - 17025 Lt. Blue - AOAC

71. 5.10 - Test Results Green - 17025 Dk. Blue - 17025/Policies & Procedures White - 17025/Records

72. Typical Findings

73. Only ISO/IEC 17025 accredited or applicant labs considered. There were a total of 7,800 cited deficiencies that was gathered from individual accreditation assessments of laboratories performed by A2LA over the last 4 years. Includes data for deficiencies against the clauses of ISO/IEC 17025, relevant test methods and specific program documents (e.g. AOAC Food Testing Requirements). Criteria used

74. Top ten Deficiencies represent over 75% of all types of Non- conforrmances (NCs). 67% are technical in nature. Rest are related to lab’s management system or A2LA policies. 10% of assessments have no NCs. Average number per assessment = 7. Deficiencies for All Labs

75. Laboratories were cited for deficiencies against ISO/IEC 17025, section 4.11 Corrective action. Not establishing a policy and a procedure or designating appropriate authorities, Lacking an investigation to determine the root cause(s) and Not monitoring the results to ensure that the corrective actions taken have been effective. Tenth Most Cited Deficiency

76. Laboratories were cited for deficiencies against R103 – General Requirements: Proficiency Testing for ISO/IEC 17025 Laboratories. Lack details in documented proficiency plan. PT results are not being provided within 30 days upon receipt; and Failure to meet the minimum proficiency testing participation frequency established. Ninth Most Cited Deficiency

77. Laboratories were cited for deficiencies against ISO/IEC 17025, section 5.6: Measurement Traceability & A2LA P113 policy. Using non-accredited calibration laboratories. Lack of accreditation body endorsement on calibration certificates. Improper reference standards for in-house calibrations. Lack of procedures for the safe handling, transport, storage and use of reference standards/materials. Eighth Most Cited Deficiency

78. Laboratories were cited for deficiencies against ISO/IEC 17025, section 5.9: Assuring the Quality of Test and Calibration Results. Failure to participate in available and relevant commercial PT programs. PT Plan not complete or current and/or PT Schedule not being followed. PT Data records incomplete. Lack of quality control procedures /Failure to perform qc checks on accredited tests. Lack of corrective action response to outliers. Seventh Most Cited Deficiency

79. Laboratories were cited for deficiencies against ISO/IEC 17025, section 4.14: Internal Audits. Inadequate procedure. Lack records of findings. Not recording area of activity audited. Failure to meet audit schedule. No corrective actions for IA findings. Not verifying continued compliance with all elements of their activities. Lacking evidence of internal auditor training and qualified to perform the audit. Sixth Most Cited Deficiency

80. Laboratories were cited for deficiencies against ISO/IEC 17025, section 5.5: Equipment. Equipment not uniquely ID’ed/labeled with calibration status/calibrated before placed into service. Lacking records of calibrations & maintenance. Defective/suspect equipment not sequestered. Intermediate checks not performed after calibration, maintenance or repair. Calibration cycles have been extended and not properly documented. Fifth Most Cited Deficiency

81. Laboratories were cited for deficiencies against ISO/IEC 17025, section 4.13: Control of Records. Failure to record original observations. Failure to retain adequate records to establish an audit trail. Procedures lacking all of the required record traceability requirements. Inadequate protection of electronic records. Records with information made illegible or scratched out and/or alterations to the records were not signed or initialed. Fourth Most Cited Deficiency

82. Laboratories were cited for deficiencies against ISO/IEC 17025, section 4.3: Document Control. Lacking defined control procedure. Documents not undergoing periodic review. Use of obsolete/uncontrolled instructions. Failure to control external documents. Master list of documents not current or complete. Lacking record trail for archived documents. Documents not uniquely identified and/or did not include all the required identification. Third Most Cited Deficiency

83. Laboratories were cited for deficiencies against ISO/IEC 17025, section 5.4: Test (and Calibration) Methods and Method Validation. Modification to method not validated. Method not confirmed prior to use. Did not follow method procedure as written. Measurement uncertainty: Improper or incomplete estimate. Lacking a procedure for estimation for in-house calibrations. Failure to validate in-house software. Second Most Cited Deficiency

84. Laboratories were cited for deficiencies against Specific Program Requirements, such as AOAC requirements. Lacking records of yearly staff training on QMS. No microbiological QC activities. Table 1 equipment calibration/ verification requirements not being followed. Media records not complete. 15% of the total were in areas specific to these. #1 Most Cited Deficiency

85. Questions / Comments

86. Contact: Roger M. Brauninger Phone: 301 644 3233 Email: rbrauninger@a2la.org Email: rbrauninger@a2la.org American Association for Laboratory Accreditation 5301 Buckeystown Pike, Suite 350 Frederick, MD 21704 www.a2la.org For Further Information