1. Module 12: Quality Assurance
Global Laboratory Initiative – Xpert MTB/RIF Training Package

2. World Health Organization
Contents of this module
19 April, 2017
What is quality assurance?
Key components of Xpert MTB/RIF quality assurance programme
Training and competence assessment
Instrument verification
Method validation
Quality control and lot testing
Quality indicator monitoring
External quality assurance/assessment (EQA)

3. World Health Organization
Learning Objectives
19 April, 2017
At the end of this module, you will be able to:
Describe the function of the two internal quality controls in each Xpert MTB/RIF cartridge
Understand and apply the requirements for Xpert MTB/RIF instrument verification
List Xpert MTB/RIF quality indicators to be monitored
Understand and apply the requirements of on-site supervision
Understand and apply requirements for proficiency testing for Xpert MTB/RIF
Understand how to develop and apply a comprehensive approach to quality assurance

4. What is Quality Assurance (QA)?
“Planned and systematic activities to provide confidence that an organization fulfills requirements for quality.” [CLSI GP26-A4]
“Encompasses a range of activities that enable laboratories to achieve and maintain high levels of accuracy and proficiency despite changes in test methods and the volume of specimens tested.” [
Quality assurance is just one part of a Laboratory Quality Management System, required to ensure quality of all its processes
Continuous quality improvement is a critical concept to be adopted by laboratories

5. Key Components of Xpert MTB/RIF Quality Assurance Program
Environment
- Safe and functional
- Temperature control
Personnel
- Trained and competent staff
- Test user is documented
- Current SOPs readily available
Equipment
- Maintained and serviced
Supplies
- Uninterrupted supplies
- Appropriate transport and storage conditions
Specimens
- Good quality
Labelled with unique ID
- Completed request form
Internal quality monitoring
- Test working properly
External
Quality Assessment (EQA)
- Testing site‘s work checked by another site
Accurate and timely reporting
- Turnaround time
Results review
Accurate, reliable and timely results

6. Quality improvement cycle
The quality improvement cycle includes the four steps Plan (P), Do (D), Check (C) and Action (A)
Non-conformities identified during routine testing and quality assurance activities should be analysed, corrective actions implemented and monitored
Steps should be repeated over time to ensure continuous improvements in lab processes

7. Safe and Functional Laboratory Environment
Bio-safety requirements similar to direct sputum smear microscopy
Refer to Module 2
Secure premises for the equipment to prevent theft of the GeneXpert unit and the computer/laptop
Stable electricity supply or sufficient measures to ensure uninterrupted supply (generator, solar panels, battery/UPS backup, etc.)
Appropriate measures to prevent ambient temperature exceeding 30°C in the room where equipment will be installed (e.g. ventilation, air conditioning)
Appropriate management of biohazardous waste according to local guidelines

8. Trained and Competent Personnel
Qualification requirements for testing personnel determined by country’s Ministry of Health or other governing body
At least 1-2 staff per testing site
Basic computer literacy
Knowledge of laboratory registers
Performs testing and interprets results
Conducts routine equipment maintenance
Troubleshoots common testing problems
Documented initial training and competency assessment
Documented annual competency assessment
Refresher training as needed
Standard Operating Procedures (SOPs) must be up to date and available onsite

9. Maintained Equipment
Maintain all laboratory equipment in a good working condition
Perform instrument verification upon installation and after calibration/module replacement
Daily, weekly, monthly, and annual maintenance tasks performed
Refer to Module 9
All maintenance tasks recorded on appropriate logs
Troubleshooting testing or instrument failures
Documentation of corrective action(s)
Warranty or service contracts in place and terms and conditions adhered to
Maintain dated service records

10. Adequate Supplies and Reagents
Plan procurement based on actual test consumption data. Rotate stock to ensure that oldest material is used first
SOPs for supply and reagent procurement and inventory management
Adequate storage of Xpert MTB/RIF cartridges at recommended temperature range (2-28°C)
Daily temperature monitoring
Corrective action documentation if temperature out of range
Label all supplies and reagents with date received, and the date first opened
Any material found to be unsatisfactory should be recorded as such and removed from the laboratory immediately so it is not used
Label prepared disinfectants with name, date prepared, expiry date

11. Quality Specimens
Ensure SOPs are available and patients are instructed in providing good quality specimens
Ensure specimens are labeled correctly and request forms are completed
Reject specimens that are incorrectly labeled, leaking, in broken containers or of insufficient volume
Record the date specimens arrive in the laboratory
Monitor specimen transport times
Evaluate and record the sputum specimen quality
Specimens should be refrigerated at 2–8°C for a maximum of 10 days
If necessary, specimens can be stored at a maximum of 35°C for up to 3 days, and then refrigerated at 2–8°C for a combined maximum duration of 10 days

12. GeneXpert user rights and QA
Slide is optional, for certain audiences
GeneXpert users may be designated as admin, detailed or basic users
This designation relates to the functions and level of detail visible on the Xpert MTB/RIF software
Some quality assurance and troubleshooting functions are only available to staff with admin or detailed user access
e.g. Ct values, graphs and error codes
Ensure the correct staff have appropriate access based on their job function, e.g. staff responsible for QA and troubleshooting need detailed or admin access
Functions available to different level users can be customised according to need

13. Internal Quality Monitoring
Each cartridge contains internal controls (Sample Processing Control and Probe Check Control)
Positive and negative controls may be run according to local guidelines
Record the results of control samples, troubleshoot any unexpected results and monitor trends over time
Control strains should be from well-characterised (phenotypic and genotypic) clinical isolates
Routinely monitor quality indicators
See quality indicator monitoring slides below

14. Internal Quality Monitoring: Sample Processing Control
Each cartridge includes a Sample Processing Control (SPC), which contains non-infectious spores in the form of a dry spore cake, to verify adequate processing of MTB
Verifies that lysis of MTB has occurred
Verifies the specimen processing is adequate
Detects specimen-associated inhibition of the real-time PCR assay
SPC must be positive when the result is MTB Not Detected; and
SPC can be negative or positive when the result is MTB Detected
The test result is invalid if the SPC is negative when the test result is negative

15. Internal Quality Monitoring: Probe Check Control
The Probe Check Control is undertaken by the system before the PCR reaction starts. The system measures the fluorescence signal from the probes to monitor:
bead rehydration
reaction-tube filling
probe integrity, and
fluorescent dye stability
Results are automatically compared to the pre-established factory settings in the software
If Probe Check fails, then the test is stopped, and an Error result is obtained

16. External Quality Assessment (Assurance) (EQA)
“Inter-laboratory comparisons and other performance evaluations that may extend throughout all phases of the testing cycle, including interpretation of results; determination of individual and collective laboratory performance characteristics of examination procedures by means of inter-laboratory comparison; NOTE: the primary objectives of EQA are educational and may be supported by additional elements.” [CLSI GP27-A2]
EQA:
Is used to improve performance across the laboratory network
Is an important tool for communicating with and motivating staff
Is designed to identify and resolve problems
Is not used to punish staff

17. Accurate and Timely Reporting
Report Xpert MTB/RIF results within 24 hours after the specimen is received at the testing site to allow rapid treatment initiation
Appropriate sample transportation systems are required to ensure rapid shipment of samples and the reporting of results to referring sites
Record all work performed in standard format in the Laboratory Register
Analyze Xpert MTB/RIF results each month to detect changes which may indicate problems
See recommended quality indicators

18. Quality Assurance Requirements
GLI recommends the following components for a comprehensive approach to quality assurance of the Xpert MTB/RIF assay:
Training and user competence assessment
Instrument verification
Quality indicator monitoring
On-site supervision
New lot (incoming) QC testing
Proficiency testing (PT)

19. Quality Assurance Requirements
Blinded re-checking (recommended for TB smear microscopy EQA) is not appropriate for Xpert MTB/RIF testing since the entire specimen is usually required to perform the test and no specimen can be saved for later re-checking
If Xpert MTB/RIF is used according to manufacturer’s specifications, method validation is not required
Use of an alternative protocol or sample that is not included in the product insert should be validated by the laboratory prior to use

20. Instrument Verification
Each module should be verified as being “fit for purpose” using known positive and/or negative material prior to starting testing of clinical specimens
At least one verification test should be performed per module after:
Instrument installation
Post-calibration or swapping of instrument modules
Verification panels are now distributed routinely by Cepheid with each new instrument and with recalibrated modules
Verification panels consist of a card containing 5 Dried Culture Spots (DCS) of a known concentration of whole inactivated Mycobacterium tuberculosis (rifampicin sensitive) bacilli
DCS samples should be processed according to instructions and one sample tested per module. All results are expected to be MTB Detected, RIF resistance Not Detected

21. Instrument Verification
If an invalid/error/no result is obtained on any module, repeat the test in that module using the extra DCS sample provided
In case of not obtaining the expected result in any module, refer to the Troubleshooting module (Module 9)
Instrument verification results should be reported to the designated GeneXpert supervisor
Cepheid should be contacted immediately in order to assist with any problems in the verification process

22. On-site Supervision
Site visits should be planned at regular intervals to assess laboratory/testing site practices
Usually conducted by NRL/NTP and/or partners
May be conducted by national level or regional/district level staff
Should be integrated with other on-site supervision where possible (e.g. quarterly NTP site visits)
Standardized checklist must be utilized for consistency and completeness of information
Identifies which practices are ‘good’ and which areas need improving
Collect information for
Planning & implementation
Monitoring
Continuous quality improvement
Especially critical during early stages of implementation of new technology
Provides motivation and support to staff, especially in peripheral settings
Strong relationships with staff encourages rapid reporting of any problems
Enables rapid troubleshooting, re-training and corrective actions

23. On-site Supervision
A schedule for site visits should be drawn up in advance, preferably integrated with other supervision activities
Responsibilities for on-site supervision may be decentralised to regional/district staff
All staff conducting supervision visits need appropriate training and should use standardised checklists
Reports should be shared with the testing site and NTRL/NTP according to local practices
On-site supervision should be conducted in conjunction with collection and review of quality indicators
Proficiency testing and monitoring of quality indicators do not negate the need for on-site supervision
Poorly performing testing sites should be prioritised for on-site visits

24. On-site Supervision
Sufficient time should be allocated for supervisory visits, including travel time
All components of Xpert MTB/RIF testing and lab workflow should be evaluated
Comprehensive assessments may be conducted less frequently (e.g. annually) by expert laboratory staff, with more frequent (e.g. quarterly) visits done by district/regional supervisors or other appropriately trained staff
The extent of evaluation during each visit will depend on frequency of visit, capacity of staff and performance of laboratory (more extensive evaluation is needed in poorly performing sites)
Assess pre- and post-analytical stages (specimen collection, recording & reporting results, confirmatory testing)
Check and analyze trends in Xpert MTB/RIF indicators
Run GeneXpert software reports to view frequencies and types of errors
Mentor staff on troubleshooting and error reporting during site visits
Visits are opportunities to:
learn
discuss concerns
solve problems

25. Quality Indicator Monitoring
To be customized by each country
Quality Indicator Monitoring
Routine monitoring of quality indicators (performance indicators) is a critical element of quality assurance for any diagnostic test and is also an ISO requirement
Every month, each testing site should collect and analyse quality indicators
Any unexplained change in quality indicators (such as increase in error rates, change in MTB positivity rate or RIF resistance rate or significant change in volume of tests conducted) should be documented and investigated
Error rates higher than a pre-determined threshold (e.g. 5%) should be investigated
Quality indicators should be reviewed by the laboratory manager and must always be linked to corrective actions if any unexpected results or trends are observed
A standard set of quality indicators should be used for all sites conducting Xpert MTB/RIF testing
A system should be in place for centralised reporting of monthly quality indicators
Documentation of corrective actions, and subsequent improvement/normalisation of lab indicators following corrective action are critical

26. Quality Indicator Monitoring
To be customized by each country
Each instrument should be monitored on a monthly basis using the following minimum set of indicators to evaluate proper use:
Number of tests performed per month
Number and proportion of MTB detected, RIF resistance not detected
Number and proportion of MTB detected, RIF resistance detected
Number and proportion of MTB detected RIF indeterminate
Number and proportion of MTB not detected
Number and proportion of errors
Number and proportion of invalid results
Number and proportion of no results
Where possible, countries should collect disaggregated data according to the population group tested (HIV positive, MDR risk, extrapulmonary)

27. Quality Indicator Monitoring
Identifying the number and type of various errors can help with troubleshooting since certain errors may be associated with processing, instrument or environmental conditions
The following analyses may be performed:
The number of errors occurring by instrument module
If a particular module produces more errors over time compared with other modules, it may require repair
The number of errors occurring by user
If a user has a high number of errors, investigation of the specific error types is necessary since some errors are due to incorrect specimen processing
The number of tests lost due to power outages or surges
The number, duration and causes of routine interruptions in the Xpert MTB/RIF testing service
Common causes are cartridge stock-outs or expiration, staff not available, instrument/computer breakdown

28. Monthly Report Xpert Results
To be customized by each country
Where possible, countries should collect disaggregated data according to the population group tested (e.g. HIV positive, MDR risk, extrapulmonary)

29. Remote Monitoring
Several systems are being piloted for remote monitoring of GeneXpert instruments
Open-source and proprietary options available
Cloud-based servers and in-country servers possible
Connectivity via internet or SMS
Data accessed via web-based dashboards
Remote monitoring can assist with collection of data relating to quality assurance, procurement and some solutions can be linked to reporting of patient results
Remote monitoring allows near real-time data to be available simultaneously to different types and levels of staff

30. New lot (incoming batch) QC testing
Incoming batches of Xpert MTB/RIF cartridges should be tested using a sample of cartridges and at least one known positive and negative specimen, to ensure expected performance
QC testing may be centralized at a reference laboratory before distribution of a new batch of cartridges to sites
In addition to new lot QC testing, continuous monitoring at site level of performance indicators of tests, including error rates, is important for the early detection of any problems with cartridge batches due to local storage conditions or other factors

31. Proficiency Testing
“A program in which multiple specimens are periodically sent to members of a group of laboratories for analysis and/or identification, in which each laboratory’s results are compared with those of other laboratories in the group and/or with an assigned value, and reported to the participating laboratory and others.” [CLSI GP27-A2]
Ideally, a PT program checks key pre-analytical, analytical, and post-analytical processes occurring in the testing site.
A number of samples are sent to the laboratory or testing site several times per year
Testing site performs Xpert MTB/RIF as they would with routine patient specimens and report results.
Results indicate quality of personnel performance and test site operations.
Results are compared to expected results and across several testing sites.
Results are monitored for trends over time.
PT does not measure routine laboratory performance but may identify laboratories with major deficiencies

32. Proficiency Testing
PT is recommended at least once per year
Feedback of PT results should be provided in a timely manner to the testing sites and supervisory staff
Rapid feedback is needed to enable prompt initiation of correction actions
While on-site supervision and routine monitoring of quality indicators are the most critical components of QA, PT helps to:
identify major non-conformities
target the most poorly performing laboratories for on-site supervision
Can be used to evaluate technician performance post-training

33. Proficiency Testing
Several options (different formats) have been evaluated and all were found to be acceptable formats
For examples, see Scott et al. J. Clin. Microbiol :
National reference laboratories may consider preparing their own PT panels in place of an external PT scheme

34. Summary Quality assurance includes all the following aspects:
Safe and functional laboratory environment, trained and competent personnel, maintained equipment, adequate supplies and reagents, testing of quality specimens, internal quality monitoring, External Quality Assessment (EQA), and accurate and timely reporting
A comprehensive quality assurance programme should include the following, and which are needed to fulfil ISO requirements:
Instrument verification using dried culture spot panels
Quality indicator monitoring
Regular on-site supervision
New lot QC testing
Proficiency testing

35. Summary
Quality assurance is part of the routine workload and is not a separate activity
All quality assurance activities must be documented
Feedback to testing sites and implementing corrective and preventive measures are the most critical aspects of any quality assurance programme
Quality assurance is needed whether Xpert is placed in a laboratory or non-laboratory setting

36. Assessment
Which quality assurance activities are recommended for Xpert MTB/RIF?
How does your country address these requirements?
What role does quality indicator monitoring play in quality assurance?
Can proficiency testing be done in place of on-site supervision?
How would you plan an on-site supervision schedule for sites performing Xpert MTB/RIF in your country?
What should happen to results of quality assurance activities?

37. Acknowledgements
The Xpert MTB/RIF Training Package has been developed by a consortium of GLI partners, including FIND, KNCV, US CDC, USAID and WHO, with funding from USAID (TB CARE I). The modules are based on materials originally developed by FIND, KNCV and Cepheid.