NJHIV's Rapid Testing Program Keeps False Positives in Check Evan M. Cadoff, MD1; Eugene G. Martin, PhD1; Sindy M. Paul, MD, MPH2; Sharon A. Holswade, MBA1; Jake Nelson1; Vivian H. Shih, BA1; Dolores Van Pelt, MSN1; Nisha J. Intwala, BS (MT)1; Franchesca N. Jackson, BS1; Gratian Salaru, MD1 1 UMDNJ-RWJMS, New Brunswick, NJ and 2 New Jersey Department of Health and Senior Services, Trenton, NJ. Abstract Background:FDA approval of rapid HIV testing kits is partly based on sensitivity and specificity claims. Recently, in San Francisco, New York and Los Angeles, health care workers confronted a spike of false positive results (positive rapid HIV test with negative confirmatory testing) on oral fluid (OMT). By reviewing NJHIV quality assurance (QA) monitoring data, we were able to assess the prevalence and significance of false positive results in NJ. Methods:The NJHIV program began rapid testing under a medical school QA program in late 2003. The program has expanded to include 117 sites by 2005. A critical factor in the organization of testing was the use of a centralized core staff of clinical laboratorians who were responsible for development of common, uniform policies, procedures, training and recertification, inventory control and validation, proficiency testing, and a core communication hub (www.njhiv.org) all part of a rigorous quality assurance initiative. Data from all sites is analyzed monthly. Results:We reviewed NJHIV data since March 2005, including over 24,000 tests, of which more than 5000 were OMT. Specificity of fingerstick testing exceeded 99.9% . A slight increase in the false positive rate (99.5% specificity) was noted when sites moved from blood to OMT testing, but specificity remained within manufacturer’s claims. Reviewing data from each of our sites, each lot of reagent, each month of testing and each specimen type, we did not record an out of label FP spike as was seen in some other states. Conclusion:Data routinely collected in our QA program allowed rapid assessment of HIV testing performance during the introduction of oral mucosal testing. No increase in false positives occurred beyond what would be expected with the transition to oral fluid testing. This is largely attributable to the organization of the program. Figure 1 OMT: A New Form of Rapid Testing Introduced NJHIV participated in a CDC sponsored evaluation of OMT testing through May 2005. OMT testing was introduced at New Jersey testing sites in June 2005. OraQuick’s specificity claim (lower bound of confidence interval) for fingerstick testing is 99.8% and for OMT testing is 99.6%. There was a gradual transition from blood to OMT testing (Figure 2). Figure 2: Rapid HIV sample types S.F. Clinics Getting High False- Positive Rate on Oral HIV Test Sabin Russell Friday, December 9, 2005 “A promising new oral HIV test being considered for home use has produced at least 47 false positives at San Francisco public health clinics, throwing a scare into those who received the results and raising questions about the test's suitability for widespread use in the United States and abroad. The OraQuick ADVANCE HIV test approved for professional use by the Food and Drug Administration in March 2004 detects, in just 20 minutes, antibodies to the virus that causes AIDS, using fluid swabbed from the mouth. Its speed and ease of use make the test a particularly suitable candidate for HIV prevention and treatment efforts that stress frequent testing and speedy access to care for those who test positive. San Francisco Department of Public Health officials were alarmed last week when their analyses of more than 6,000 oral tests since Spring turned up 47 instances in which the results were positive, but follow-up tests showed those patients were not infected.” PRESS RELEASE AVALANCHE “SF Clinics Getting High False Positive Rates on Oral HIV Test” San Francisco Chronicle 12/10/05 “False Positives from HIV Test” New York Times 12/11/05 “Facility Halts Use of Oral HIV Test” LA Times 12/16/05 “More Sites Drop Oral HIV Test” LA Times 12/20/05 Methods NJHIV testing sites maintain logs of each test performed (Figure 3). Figure 3: NJHIV testing log Data from the logs is entered into an Excel spreadsheet (Figure 4). Data from the logs is compared to the NJ Counseling and Testing database and the database of confirmatory tests run at the NJDHSS - Public Health and Environmental Laboratories. The NJHIV database was queried for discordant results and reviewed by month, by testing site, by OraQuick reagent lot number, and by sample matrix--ie, blood or oral fluid. (Figures 4a and 4b). Results Data comparison among databases showed that data was complete in the NJHIV database for 22 of 23 sites tested. After remediation of recordkeeping practices, NJHIV data at all sites is complete through June 2006. One cluster of 3 simultaneous discordants was discovered at one site. Further analysis did not yield an explanation for these discordants. Consistent with the manufacturer’s product performance claims, there was a slightly higher false positive rate when using oral fluid than when using fingerstick whole blood specimens (Figure 5). Specificity occasionaly drops below manufacturer’s claims for individual months. Figure 5: Specificity by month Analysis of discordant results did not show any other pattern when examined by month, site, reagent lot or sample type. November 2003 1st site Rollout of primary sites April 2004 28 primary sites QA plan adapted to fixed satellite sites June 2005 102 fixed satellites Temperature, lighting, transportation issues resolved, to support large mobile vans and converted minivans November 2005 10 mobile vans March 2006 140 licensed Sites include: CTS sites, Emergency Departments, HIV prevention programs, hospital inpatients,hospital outpatient clinics, STD clinics TB clinics, Faith-based initiatives, mobile vans CDC's PMS-2 study begins July 2004 December 2005 PMS-2 ends October 2004 CDC's OMT study begins May 2005 OMT ends Expansion of rapid HIV testing sites, and participation in CDC-sponsored studies Conclusions Our quality assurance system and databases allowed us to quickly evaluate our test results in response to reports of increased numbers of false positive tests in San Francisco, New York and Los Angeles. False positive OraQuick results in New Jersey were not an issue. Specificity remained within the claims of the manufacturer. The CDC presented NJHIV’s data at national meetings (Figure 6) and included our findings in their decision not to change current rapid test recommendations. Figure 6: Slide from CDC presentation at CROI meeting, February 2006 Background The NJHIV Rapid HIV Testing Program Built upon existing UMDNJ-Robert Wood Johnson Medical School, multi-facility, point-of-care-testing program. Centralized quality assurance process using pathologists, technologists and informaticians. OraQuick fingerstick whole blood testing began November 2003 at the first NJ licensed site – a primary site in close proximity to the medical school. Currently there are 140 sites including fixed and mobile venues. (Figure 1) Testing activities are reviewed monthly by a medical technologist. Preliminary positive rapid tests are confirmed by Western Blot at the NJDHSS – PHEL laboratory in Trenton. Quality Assurance Plan Management by a board certified Pathologist Supervisory control through site coordinators Central lab oversees: Regulatory issues and proficiency testing Acquisition and validation of supplies Inventory control Common procedures and core policies Uniform administration at all locations Common training, certification of personnel, forms Core communication hub www.njhiv.org Quality Control Rules Standardized monthly site visits – ‘The Report Card’ Intra and inter site comparisons to insure that requirements for quality and process control are maintained Figure 4: Test records entered into database which tracks frequency of testing by month, primary site, satellite, reagent lot numbers, sample type and test result. Figure 4a: Records selected for review based on a single parameter. All oral fluid tests were selected in the diagram to the left and are shown here. Figure 4b: Records selected for review based on a second parameter. Here, all discordant results were selected from among the oral fluid tests in the diagram to the left.