1 Module OVERVIEW OF EXTERNAL QUALITY ASSESSMENT

2 Content Overview What is EQA? EQA for AFB Smear Microscopy Panel testing Blinded rechecking On-site evaluation Advantages and disadvantages of EQA methods Identification of the most feasible EQA method for a country

3 External Quality Assessment (EQA) A system for objectively checking the laboratory’s performance using an external agency or facility Ensures customers (physicians, patients and health authorities) that the laboratory can produce reliable results An indispensable part of a laboratory quality management system

4 The Quality System EXTERNAL QUALITY ASSESSMENT Quality System Essentials (QSE) Set of coordinated activities that function as building blocks for quality management Quality System Essentials (QSE) Set of coordinated activities that function as building blocks for quality management

5 ISO 15189:2007 Requirements Regarding EQA The laboratory should participate in interlaboratory comparisons such as those organized by EQA schemes The lab management shall monitor the results of EQA and participate in the implementation of corrective actions

6 EQA Benefits Allows inter-lab comparison of performance Serves as an early warning system Identifies systematic problems Provides objective evidence of laboratory quality Serves as an indicator for focusing improvement efforts Identifies training needs Source of continuing education Source of material for practice

7 EQA for AFB Smear Microscopy Consensus document: External Quality Assessment for AFB Smear Microscopy (2002)

8 EQA Methods for AFB Smear Microscopy External Quality Assessment (EQA) – a process to assess laboratory performance; allows to assess labs’ capabilities and performances by comparing their results with those obtained in other labs in the network On-site supervision Panel testing Blinded rechecking

9 Panel Testing Process NRL sends out sets of stained and/or unstained sputum smears for testing Laboratory technicians analyze smears and return results to NRL Results are evaluated, scores are sent to participants Appropriate corrective actions are undertaken (and documented), if needed

10 National reference laboratory Intermediate laboratory Peripheral laboratories A POSSIBLE SCHEME OF A PANEL TESTING ROUND Peripheral laboratories

11 Blinded Rechecking Random sampling of routine slides from a peripheral laboratory for rechecking by a higher-level laboratory The widely used system for rechecking of “10% of negative and 100% of positive smear” is no more recommended The proposed blinded rechecking method is based on the Lot Quality Assurance System (LQAS)

12 National reference laboratory Peripheral laboratories Intermediate laboratory SECOND CONTROLLER Routine slides: random sampling Discordant slides FIRST CONTROLLER Routine slides: random sampling A Proposed Scheme of Organizing Blinded Rechecking for AFB Smear Microscopy A Proposed Scheme of Organizing Blinded Rechecking for AFB Smear Microscopy

13 On-Site Supervision Periodic visits to the laboratory to assess laboratory practices to: Obtain a realistic picture of laboratory practices Provide assistance with problem areas, including training Laboratory vs. TB supervisors When planning site visits, to consider: Frequency Use of checklists Follow-up visits Monitoring corrective actions Training

14 EQA methods: Which one is better? Advantages and drawbacks for all types Influencing factors: NTP goals set up for laboratory services development TB prevalence in the country Centralization / decentralization of health services Resources available and projected Cost, time Logistical questions

15 Panel Testing ADVANTAGESDISADVANTAGES: Low workload for a peripheral center Improves laboratory credibility Rapid response countrywide possible Use of stained and unstained smears can help to identify the source of a problem May lead to identification of faulty equipment Does not measure routine performance High workload for NRL May not be motivating to improve daily performance

16 Minimal first step for EQA with limited resources Rapid assessment of gross deficiencies Evaluates proficiency of laboratory technicians prior to and following training A tool during problem-oriented supervision Panel Testing: Indications for Use

17 Blinded Rechecking ADVANTAGESDISADVANTAGES: Low workload for a peripheral laboratory Motivates to improve daily performance Reflects reality of routine performance Higher workload for a higher level center Needs close adherence to elaborated procedures Can not be used with very low positivity rates Indications for use: The best method for evaluating lab performance Countrywide Ongoing and permanent

18 On-Site Supervision ADVANTAGESDISADVANTAGES: Direct personal contact Motivating to staff Observation of actual work Identifies causes of errors Permits verification of equipment quality and function Usually poor coverage Labor intensive Costly Needs very good supervisors

19 On-Site Supervision: Indications for Use Complementary to rechecking and panel testing for constructive feedback and problem solving Implementation and monitoring of quality improvement measures Data collection and flow of information among laboratory levels

20 Blinded rechecking or panel testing should be complemented by very targeted problem- oriented supervision conducted by a trained staff EQA: Important Issue

21 Situational Analysis: How to Identify the Most Feasible EQA method(s) For a Country? (I) COUNTRY A: Populous; vast territory High burden of TB Several hundreds of geographically dispersed peripheral laboratories Intermediate laboratories’ infrastructure is not well developed; many intermediate labs experience lack of staff NRL is well equipped and staffed

22 Situational Analysis: How to Identify the Most Feasible EQA method(s) For a Country? (II) COUNTRY B: Less populated (compared to Country A); compact territory High burden of TB Smear microscopy laboratory network: one microscopy center for population High to medium positivity rates Well developed structure of intermediate laboratories, supervised by NRL Lack / rotation of staff is a typical problem for many laboratories

23 Situational Analysis: How to Identify the Most Feasible EQA method(s) For a Country? (III) COUNTRY C: Populous country Low burden of TB Only few laboratories dealing with sputum smear microscopy Low volume sputum smear examinations; very low positivity rates AFB smear-positive specimens are rechecked at NRL

24 Key Messages (I): EQA can be defined as a system for objectively checking the laboratory’s performance using an external agency or facility EQA for AFB smear microscopy allows participating laboratories to access their capabilities and performances by comparing their results with those in other laboratories in the network EQA for AFB sputum smear microscopy include three methods: Panel testing Blinded rechecking On-site supervision

25 Key Messages (II): Effective use of each of EQA methods will highly depend on resources, NTP goals, TB prevalence, health services’ structure and the stage of laboratory quality assurance activities development in the country Blinded rechecking or panel testing should be complemented by very targeted problem- oriented supervision conducted by a trained staff