1. NJHIV's Rapid Testing Program Keeps False Positives in Check
Evan M. Cadoff, MD1; Eugene G. Martin, PhD1; Sindy M. Paul, MD, MPH2; Sharon A. Holswade, MBA1; Jake Nelson1; Vivian H. Shih, BA1; Dolores Van Pelt, MSN1;
Nisha J. Intwala, BS (MT)1; Franchesca N. Jackson, BS1; Gratian Salaru, MD1
1 UMDNJ-RWJMS, New Brunswick, NJ and 2 New Jersey Department of Health and Senior Services, Trenton, NJ.
Abstract
Background:FDA approval of rapid HIV testing kits is partly based on sensitivity and specificity claims. Recently, in San Francisco, New York and Los Angeles, health care workers confronted a spike of false positive results (positive rapid HIV test with negative confirmatory testing) on oral fluid (OMT). By reviewing NJHIV quality assurance (QA) monitoring data, we were able to assess the prevalence and significance of false positive results in NJ.
Methods:The NJHIV program began rapid testing under a medical school QA program in late The program has expanded to include 117 sites by A critical factor in the organization of testing was the use of a centralized core staff of clinical laboratorians who were responsible for development of common, uniform policies, procedures, training and recertification, inventory control and validation, proficiency testing, and a core communication hub ( all part of a rigorous quality assurance initiative. Data from all sites is analyzed monthly.
Results:We reviewed NJHIV data since March 2005, including over 24,000 tests, of which more than 5000 were OMT. Specificity of fingerstick testing exceeded 99.9% . A slight increase in the false positive rate (99.5% specificity) was noted when sites moved from blood to OMT testing, but specificity remained within manufacturer’s claims. Reviewing
data from each of our sites, each lot of reagent, each month of testing and each specimen type, we did not record an out of label FP spike as was seen in some other states.
Conclusion:Data routinely collected in our QA program allowed rapid assessment of HIV testing performance during the introduction of oral mucosal testing. No increase in false positives occurred beyond what would be expected with the transition to oral fluid testing. This is largely attributable to the organization of the program.
Figure 1
OMT: A New Form of Rapid Testing Introduced
NJHIV participated in a CDC sponsored evaluation of OMT testing through May 2005.
OMT testing was introduced at New Jersey testing sites in June 2005.
OraQuick’s specificity claim (lower bound of confidence interval) for fingerstick testing is 99.8% and for OMT testing is 99.6%.
There was a gradual transition from blood to OMT testing (Figure 2).
Figure 2:
Rapid HIV
sample types
S.F. Clinics Getting High False-
Positive Rate on Oral HIV Test
Sabin Russell
Friday, December 9, 2005
“A promising new oral HIV test being considered for home use has produced at least 47 false positives at San Francisco public health clinics, throwing a scare into those who received the results and raising questions about the test's suitability for widespread use in the United States and abroad.
The OraQuick ADVANCE HIV test approved for professional use by the Food and Drug Administration in March 2004 detects, in just 20 minutes, antibodies to the virus that causes AIDS, using fluid swabbed from the mouth. Its speed and ease of use make the test a particularly suitable candidate for HIV prevention and treatment efforts that stress frequent testing and speedy access to care for those who test positive.
San Francisco Department of Public Health officials were alarmed last week when their analyses of more than 6,000 oral tests since Spring turned up 47 instances in which the results were positive, but follow-up tests showed those patients were not infected.”
PRESS RELEASE AVALANCHE
“SF Clinics Getting High False Positive Rates on Oral HIV Test”
San Francisco Chronicle 12/10/05
“False Positives from HIV Test”
New York Times 12/11/05
“Facility Halts Use of Oral HIV Test”
LA Times 12/16/05
“More Sites Drop Oral HIV Test”
LA Times 12/20/05
Methods
NJHIV testing sites maintain logs of each test performed (Figure 3).
Figure 3:
NJHIV testing log
Data from the logs is entered into an Excel spreadsheet (Figure 4).
Data from the logs is compared to the NJ Counseling and Testing database and the database of confirmatory tests run at the NJDHSS - Public Health and Environmental Laboratories.
The NJHIV database was queried for discordant results and reviewed by month, by testing site, by OraQuick reagent lot number, and by sample matrix--ie, blood or oral fluid. (Figures 4a and 4b).
Results
Data comparison among databases showed that data was complete in the NJHIV database for 22 of 23 sites tested.
After remediation of recordkeeping practices, NJHIV data at all sites is complete through June 2006.
One cluster of 3 simultaneous discordants was discovered at one site. Further analysis did not yield an explanation for these discordants.
Consistent with the manufacturer’s product performance claims, there was a slightly higher false positive rate when using oral fluid than when using fingerstick whole blood specimens (Figure 5). Specificity occasionaly drops below manufacturer’s claims for individual months.
Figure 5:
Specificity
by month
Analysis of discordant results did not show any other pattern when examined by month, site, reagent lot or sample type.
November 2003
1st site
Rollout
of primary
sites
April 2004
28 primary sites
QA plan
adapted to
fixed satellite sites
June 2005
102 fixed satellites
Temperature, lighting, transportation
issues resolved, to support
large mobile vans and converted minivans
November 2005
10 mobile vans
March 2006
140 licensed
Sites include:
CTS sites, Emergency Departments, HIV prevention programs,
hospital inpatients,hospital outpatient clinics, STD clinics
TB clinics, Faith-based initiatives, mobile vans
CDC's PMS-2 study begins
July 2004
December 2005
PMS-2 ends
October 2004
CDC's OMT study begins
May 2005
OMT ends
Expansion of rapid HIV testing sites, and participation in CDC-sponsored studies
Conclusions
Our quality assurance system and databases allowed us to quickly evaluate our test results in response to reports of increased numbers of false positive tests in San Francisco, New York and Los Angeles.
False positive OraQuick results in New Jersey were not an issue. Specificity remained within the claims of the manufacturer.
The CDC presented NJHIV’s data at national meetings (Figure 6) and included our findings in their decision not to change current rapid test recommendations.
Figure 6: Slide from CDC presentation at CROI meeting, February 2006
Background
The NJHIV Rapid HIV Testing Program
Built upon existing UMDNJ-Robert Wood Johnson Medical School, multi-facility, point-of-care-testing program.
Centralized quality assurance process using pathologists, technologists and informaticians.
OraQuick fingerstick whole blood testing began November 2003 at the first NJ licensed site – a primary site in close proximity to the medical school. Currently there are 140 sites including fixed and mobile venues. (Figure 1)
Testing activities are reviewed monthly by a medical technologist.
Preliminary positive rapid tests are confirmed by Western Blot at the NJDHSS – PHEL laboratory in Trenton.
Quality Assurance Plan
Management by a board certified Pathologist
Supervisory control through site coordinators
Central lab oversees:
Regulatory issues and proficiency testing
Acquisition and validation of supplies
Inventory control
Common procedures and core policies
Uniform administration at all locations
Common training, certification of personnel, forms
Core communication hub
Quality Control Rules
Standardized monthly site visits – ‘The Report Card’
Intra and inter site comparisons to insure that requirements for quality and process control are maintained
Figure 4: Test records entered into database which tracks frequency of testing by month, primary site, satellite, reagent lot numbers, sample type and test result.
Figure 4a: Records selected for review based on a single parameter. All oral fluid tests were selected in the diagram to the left and are shown here.
Figure 4b: Records selected for review based on a second parameter. Here, all discordant results were selected from among the oral fluid tests in the diagram to the left.