ABSTRACT

We assessed the performance of the CoronaCHEK lateral flow assay on samples from Uganda and Baltimore to determine the impact of geographic origin on assay performance. Plasma samples from severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) PCR-positive individuals (Uganda, 78 samples from 78 individuals, and Baltimore, 266 samples from 38 individuals) and from prepandemic individuals (Uganda, 1,077, and Baltimore, 532) were evaluated. Prevalence ratios (PR) were calculated to identify factors associated with a false-positive test. After the first positive PCR in Ugandan samples, the sensitivity was 45% (95% confidence interval [CI], 24,68) at 0 to 7 days, 79% (95% CI, 64 to 91) at 8 to 14 days, and 76% (95% CI, 50 to 93) at >15 days. In samples from Baltimore, sensitivity was 39% (95% CI, 30 to 49) at 0 to 7 days, 86% (95% CI, 79 to 92) at 8 to 14 days, and 100% (95% CI, 89 to 100) at 15 days after positive PCR. The specificity of 96.5% (95% CI, 97.5 to 95.2) in Ugandan samples was significantly lower than that in samples from Baltimore, 99.3% (95% CI, 98.1 to 99.8; P < 0.01). In Ugandan samples, individuals with a false-positive result were more likely to be male (PR, 2.04; 95% CI, 1.03,3.69) or individuals who had had a fever more than a month prior to sample acquisition (PR, 2.87; 95% CI, 1.12 to 7.35). Sensitivity of the CoronaCHEK was similar in samples from Uganda and Baltimore. The specificity was significantly lower in Ugandan samples than in Baltimore samples. False-positive results in Ugandan samples appear to correlate with a recent history of a febrile illness, potentially indicative of a cross-reactive immune response in individuals from East Africa.

INTRODUCTION

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection causes coronavirus disease 2019 (COVID-19) (1), which has been detected on all continents and continues to be a public health emergency globally (2). Critical to public health efforts to combat the pandemic are accurate serologic assays to differentiate exposed from unexposed individuals (3). Many studies investigate the performance of these assays on samples from Asia (4), Western Europe (5), and the United States (6). However, little information is available on the performance of these assays in an African setting, though initial studies provide evidence of potential problems (7), particularly among febrile patients infected by other infectious pathogens (8).
Serologic assays used for the detection of antibodies to different viral infections can vary in performance based on the origin of the samples being tested, as has been seen with HIV (9), hepatitis C virus (HCV) (10), and herpes simplex virus 2 (HSV-2) (11). It is thought that these differences in specificity result from host genetics of the source population and the frequency and distribution of the infectious agents the population is exposed to (12). We sought to compare the performance of the CoronaCHEK lateral flow assay (LFA) on samples from Uganda and the United States to assess the impact of geographic origin on the performance of this assay. Samples from known SARS-CoV-2-infected individuals with known duration of infection and prepandemic samples were tested to evaluate the sensitivity and specificity of the assay and to identify factors associated with a false-positive result.

MATERIALS AND METHODS

Ethics statement.

The use of samples from Baltimore was approved by The Johns Hopkins University School of Medicine Institutional Review Board (IRB00247886, IRB00250798, and IRB00091667). The use of samples from Uganda was approved by the Uganda Virus Research Institute’s Research Ethics Committee (GC/127/20/04/773 and GC/127/13/01/16), Western Institutional Review Board, protocol 200313317, and the Uganda National Council for Science and Technology (HS637ES). The parent studies were conducted according to the ethical standards of the Helsinki Declaration of the World Medical Association, where all subjects provided written informed consent. All samples were deidentified prior to testing.

Sample sets.

To assess sensitivity, samples from subjects known to be SARS-CoV-2 PCR+ in Uganda and the United States with known duration from first PCR+ date were evaluated. Samples from 78 PCR+ individuals at different time intervals were identified at the Uganda Virus Research Institute in Entebbe and at Makerere University in Kampala, Uganda. None of the Ugandan individuals were hospitalized, and all had mild disease. Samples (n = 266) from the United States were from 38 hospitalized COVID-19 patients attending the Johns Hopkins Hospital in Baltimore, MD, in the United States (13).
To assess the specificity of the assay, prepandemic samples were tested. This included 1,077 stored samples from the Rakai Community Cohort Study, collected between 2011 and 2013 (14). The Ugandan samples included 543 individuals who reported having been febrile within the month prior to sample acquisition and 534 individuals who did not report a febrile illness, matched by age and gender. The 532 prepandemic samples from the United States were remnant plasma samples collected in the Johns Hopkins Hospital Emergency Department (JHH ED) between December 2015 and January 2016 (15).

Laboratory testing and statistical analysis.

All samples were analyzed with the CoronaCHEK LFA (Hangzhou Biotest Biotech Co. Ltd.) according to the manufacturer’s protocol. CoronaCHEK contains separate IgM and IgG bands and uses the spike receptor binding domain (RBD) as the target antigen. The manufacturer reports a combined sensitivity of 100% (confidence interval [CI], 88.7 to 100) and a combined specificity of 100% (CI, 95.4 to 100). Sensitivity by duration of infection and specificity among prepandemic samples were assessed for the presence of either IgM or IgG bands for any reactivity. Statistical analysis was performed with STATA 14.2 (StataCorp, College Station, TX, USA), and 95% CI for sensitivity and specificity were calculated with the Clopper-Pearson exact method. Bivariate Poisson regression models were used to calculate prevalence ratios (PR) for factors associated with a false-positive test among prepandemic samples.

RESULTS

There were significant differences in the performance for the CoronaCHEK LFA between samples from Uganda and Baltimore (Table 1). When any reactivity was compared (IgM or IgG), there was no significant difference in reactivity by duration of infection. Though 100% of samples from Baltimore were seropositive by 14 days after their first time point, this was not the case for the Ugandan samples. Specificity, when any reactive band was considered a false-positive result, was significantly lower in Ugandan samples at 96.9% (CI, 95.2 to 97.5) than in those from Baltimore, at 99.3% (CI, 98.1 to 99.8; P < 0.01).
TABLE 1
TABLE 1 Sensitivity and specificity of the CoronaCHEK lateral flow point-of-care assay for the detection of IgM and IgG antibodies to SARS-CoV-2
Parameter (n)% (95% CI) for:
IgMIgGIgM or IgG
Sensitivity   
 Uganda   
  ≤7 days (22)41 (21–64)41 (21–64)45 (24–68)
  >7 to 14 days (39)74 (58–87)49 (32–87)79 (64–91)
  >14–28 days (17)41 (18–67)65 (38–86)76 (50–93)
 Baltimore   
  ≤7 days (102)34 (25–44)21 (13–30)39 (30–49)
  >7 to 14 days (132)82 (74–88)75 (67–82)86 (79–92)
  >14–28 days (32)100 (89–100)100 (89–100)100 (89–100)
Specificity   
 Uganda (1,077)96.9 (95.7–97.9)99.4 (98.7–99.7)96.5 (95.2–97.5)
 Baltimore (532)99.3 (98.1–99.8)100 (99.3–100)99.3 (98.1–99.8)
There were 4 and 38 false-positive results in Baltimore prepandemic samples and Ugandan samples, respectively. All four from Baltimore were all faint IgM bands, while 82% (31/38) of the false-positive samples from Uganda had only reactive IgM bands. Of the seven prepandemic Ugandan samples that were IgG reactive, two were also reactive for IgM. Ugandan samples were significantly more likely to be misclassified if they came from men (PR, 2.04; 95% CI, 1.03 to 3.69; P = 0.04) or if the individual had reported fever more than a month prior to sample collection (PR, 2.87; 95% CI, 1.12 to 7.35; P = 0.028). There was a trend to test positive if they had reported pneumonia-like symptoms (PR, 2.34; 95% CI, 0.98 to 5.59; P = 0.056). Other factors not associated with a false-positive result included age, community type, and HIV status (Table 2). There were too few misclassified samples from Baltimore to assess factors associated with misclassification within this population.
TABLE 2
TABLE 2 Factors associated with a false-positive SARS-CoV-2 antibody response in samples from Uganda
Defining categorySARS-CoV-2 antibody-positive outcome
% (no./total)PR (95% CI)
Sex  
 Female2.7 (20/737)Reference
 Male5.3 (18/340)2.06 (1.03 to 3.69)
Age  
 18–243.1 (10/327)Reference
 25–344.3 (19/439)1.42 (0.66 to 3.04)
 35–442.7 (7/260)0.88 (0.34 to 2.31)
 45–543.9 (2/61)1.28 (0.28 to 5.85)
Community type  
 Agrarian3.2 (14/436)Reference
 Fishing5.1 (19/372)1.59 (0.80 to 3.17)
 Trading1.9 (5/269)0.58 (0.21 to 1.61)
Pregnancy (no males in analysis)  
 Not pregnant2.5 (8/318)Reference
 Pregnant2.9 (12/419)1.14 (0.47 to 2.78)
Fever < 1 mo  
 No3.2 (17/534)Reference
 Yes3.9 (21/543)1.21 (0.64 to 2.30)
Fever > 1 mo  
 No3.2 (33/1,023)Reference
 Yes9.3 (5/54)2.87 (1.12 to 7.35)
Cough  
 No3.3 (27/825)Reference
 Yes4.4 (11/252)1.33 (0.66 to 2.69)
Pneumonia  
 No3.2 (32/997)Reference
 Yes7.5 (6/80)2.34 (0.98 to 5.59)
HIV status  
 Negative3.4 (21/618)Reference
 Positive3.7 (17/459)1.09 (0.58 to 2.07)

DISCUSSION

This study demonstrates differential performance of the CoronaCHEK LFA on samples collected from Uganda compared to those collected from Baltimore. Though sensitivity for both IgG and IgM in samples from Baltimore was 100% by 14 days after the subjects’ first PCR+ date, unlike samples from Uganda, this difference was not significantly different. Though not significant, there was a substantial difference in the point estimates, with the Ugandan samples having a sensitivity of only 76% at >14 days after the subjects’ first PCR+ date. These differences in sensitivity could be due to Ugandan samples coming from individuals with mild disease while the samples from Baltimore were from hospitalized individuals. Specificity was significantly lower in the Ugandan prepandemic samples than those from Baltimore, though this difference was all associated with the IgM band. False-positive results in Ugandan samples were higher among men and those who had reported a febrile episode more than a month prior to sample acquisition. Of the false-positive results detected, the vast majority were IgM reactive.
These results demonstrate that the performance characteristics of serological assays for SARS-CoV-2 antibody detection cannot be extrapolated to different populations without adequate validation studies. This study supports the need for validation studies on SARS-CoV-2 serologic assays in Africa, an area for which few data exist (16). Though a lower specificity was found in Ugandan samples than those from Baltimore, the specificity of 96.5% was much greater than the 85% found for the Euroimmun IgG S1 enzyme-linked immunosorbent assay (ELISA) in prepandemic samples from Benin (8). As shown in the study by Mboumba Bouassa (7), our study demonstrated that the main cause for false-positive results was a reactive IgM test. If one ignores the presence of an IgM band, the specificity of the CoronaCHEK increased to 99.4% (95% CI, 98.7 to 99.7) for Ugandan samples and 100% (95% CI, 99.3 for 100) for Baltimore samples, with no loss of sensitivity at 14 days after the first positive PCR for SARS-CoV-2.
There are a number of limitations of our study. First, the samples from Uganda of SARS-CoV-2-infected patients were limited, with only six samples within the first week after the first PCR-positive test and no serial samples for a given individual. Additionally, these samples came from known infected Ugandan individuals with limited symptoms, while the Baltimore samples all came from known SARS-CoV-2-positive individuals who were hospitalized. The prepandemic samples from Baltimore were not matched to those from Uganda based on symptomology, though historically, individuals attending the ED in the United States have a high prevalence of fever and viral infections (17). Samples from the JHH ED do have a high burden of chronic viral infections, as demonstrated by seroprevalences of 6%, 12%, and 50% for HIV, HCV, and HSV-2, respectively (18).
In summary, the geographical origin of the samples appeared to impact the performance of the CoronaCHEK LFA. IgM reactivity was the main cause for the false-positive results. Since IgM responses generally appear a couple of days before IgG responses, it may be useful not to measure IgM at all in serological studies, given the improvement in specificity. Further evaluations of serologic assays are needed to find appropriate tools for serosurveillance in an African setting.

ACKNOWLEDGMENTS

We acknowledge all of the participants who contributed specimens to this study and the study staff, without whom this study would not have been possible.
We have no conflicts of interest to declare.
This study was supported by the Division of Intramural Research, National Institute of Allergy and Infectious Diseases (NIAID), National Institutes of Health (NIH), as well as by extramural support from NIAID UM1-AI068613 to R.E.F. and NIH Center of Excellence in Influenza Research and Surveillance HHSN272201400007C to R.E.R.

REFERENCES

1.
Zhu N, Zhang D, Wang W, Li X, Yang B, Song J, Zhao X, Huang B, Shi W, Lu R, Niu P, Zhan F, Ma X, Wang D, Xu W, Wu G, Gao GF, Tan W, China Novel Coronavirus Investigating and Research Team. 2020. A novel coronavirus from patients with pneumonia in China, 2019. N Engl J Med 382:727–733.
2.
Eurosurveillance Editorial Team. 2020. Note from the editors: World Health Organization declares novel coronavirus (2019-nCoV) sixth public health emergency of international concern. Euro Surveill 25:200131e.
3.
Bermingham WH, Wilding T, Beck S, Huissoon A. 2020. SARS-CoV-2 serology: test, test, test, but interpret with caution! Clin Med J R Coll Physicians London 20:365–368.
4.
Zhao J, Yuan Q, Wang H, Liu W, Liao X, Su Y, Wang X, Yuan J, Li T, Li J, Qian S, Hong C, Wang F, Liu Y, Wang Z, He Q, Li Z, He B, Zhang T, Fu Y, Ge S, Liu L, Zhang J, Xia N, Zhang Z. 2020. Antibody responses to SARS-CoV-2 in patients with novel coronavirus disease 2019. Clin Infect Dis 71:2027–2034.
5.
Sacristan MS, Collazos-Blanco A, Cintas MIZ, García AS, de Villavicencio CY, Maestre MM. 2020. Comparison of various serological assays for novel SARS-COV-2. Eur J Clin Microbiol Infect Dis 2020:1–6.
6.
Patel EU, Bloch EM, Clarke W, Hsieh YH, Boon D, Eby Y, Fernandez RE, Baker OR, Keruly M, Kirby CS, Klock E, Littlefield K, Miller J, Schmidt HA, Sullivan P, Piwowar-Manning E, Shrestha R, Redd AD, Rothman RE, Sullivan D, Shoham S, Casadevall A, Quinn TC, Pekosz A, Tobian AAR, Laeyendecker O. 2020. Comparative performance of five commercially available serologic assays to detect antibodies to SARS-CoV-2 and identify individuals with high neutralizing titers. J Clin Microbiol 59:e02257-20.
7.
Mboumba Bouassa RS, Péré H, Tonen-Wolyec S, Longo JDD, Moussa S, Mbopi-Keou FX, Mossoro-Kpinde CD, Grésenguet G, Veyer D, Bélec L. 2021. Unexpected high frequency of unspecific reactivities by testing pre-epidemic blood specimens from Europe and Africa with SARS-CoV-2 IgG–IgM antibody rapid tests points to IgM as the Achilles heel. J Med Virol 93:2196–2203.
8.
Yadouleton A, Sander AL, Moreira-Soto A, Tchibozo C, Hounkanrin G, Badou Y, Fischer C, Krause N, Akogbeto P, De Oliveira Filho EF, Dossou A, Brünink S, Aïssi MAJ, Djingarey MH, Hounkpatin B, Nagel M, Drexler JF. 2021. Limited specificity of serologic tests for SARS-CoV-2 antibody detection. Emerg Infect Dis 27:233–237.
9.
Van Kerckhoven I, Vercauteren G, Piot P, Van Der Groen G. 1991. Comparative evaluation of 36 commercial assays for detecting antibodies to HIV. Bull World Health Organ 69:753–760.
10.
Mullis CE, Laeyendecker O, Reynolds SJ, Ocama P, Quinn J, Boaz I, Gray RH, Kirk GD, Thomas DL, Quinn TC, Stabinski L. 2013. High frequency of false-positive hepatitis C virus enzyme-linked immunosorbent assay in Rakai, Uganda. Clin Infect Dis 57:1747–1750.
11.
Biraro S, Mayaud P, Morrow RA, Grosskurth H, Weiss HA. 2011. Performance of commercial herpes simplex virus type-2 antibody tests using serum samples from sub-Saharan Africa: a systematic review and meta-analysis. Sex Transm Dis 38:140–147.
12.
Ng KW, Faulkner N, Cornish GH, Rosa A, Harvey R, Hussain S, Ulferts R, Earl C, Wrobel AG, Benton DJ, Roustan C, Bolland W, Thompson R, Agua-Doce A, Hobson P, Heaney J, Rickman H, Paraskevopoulou S, Houlihan CF, Thomson K, Sanchez E, Shin GY, Spyer MJ, Joshi D, O'Reilly N, Walker PA, Kjaer S, Riddell A, Moore C, Jebson BR, Wilkinson M, Marshall LR, Rosser EC, Radziszewska A, Peckham H, Ciurtin C, Wedderburn LR, Beale R, Swanton C, Gandhi S, Stockinger B, McCauley J, Gamblin SJ, McCoy LE, Cherepanov P, Nastouli E, Kassiotis G. 2020. Preexisting and de novo humoral immunity to SARS-CoV-2 in humans. Science 370:1339–1343.
13.
Conklin SE, Martin K, Manabe YC, Schmidt HA, Miller J, Keruly M, Klock E, Kirby CS, Baker OR, Fernandez RE, Eby YJ, Hardick J, Shaw-Saliba K, Rothman RE, Caturegli PP, Redd AD, Tobian AAR, Bloch EM, Benjamin Larman H, Quinn TC, Clarke W, Laeyendecker O. 2020. Evaluation of serological SARS-CoV-2 lateral flow assays for rapid point-of-care testing. J Clin Microbiol 59:e02020-20.
14.
Grabowski MK, Serwadda DM, Gray RH, Nakigozi G, Kigozi G, Kagaayi J, Ssekubugu R, Nalugoda F, Lessler J, Lutalo T, Galiwango RM, Makumbi F, Kong X, Kabatesi D, Alamo ST, Wiersma S, Sewankambo NK, Tobian AAR, Laeyendecker O, Quinn TC, Reynolds SJ, Wawer MJ, Chang LW. 2017. HIV prevention efforts and incidence of HIV in Uganda. N Engl J Med 377:2154–2166.
15.
Mohareb AM, Patel AV, Laeyendecker OB, Toerper MF, Signer D, Clarke WA, Kelen GD, Quinn TC, Haukoos JS, Rothman RE, Hsieh Y-H. 2021. The HIV screening cascade: current Emergency Department-based screening strategies leave many patients with HIV undiagnosed. J Acquir Immune Defic Syndr 87:e167–e169.
16.
Jacobs J, Kühne V, Lunguya O, Affolabi D, Hardy L, Vandenberg O. 2020. Implementing COVID-19 (SARS-CoV-2) rapid diagnostic tests in sub-Saharan Africa: a review. Front Med (Lausanne) 30:557797.
17.
Weiss AJ, Wier LM, Stocks C, Blanchard J. 2006. Overview of emergency department visits in the United States, 2011: Statistical Brief #174, Healthcare Cost and Utilization Project (HCUP) statistical briefs. Agency for Healthcare Research and Quality, Rockville, MD.
18.
Patel EU, Laeyendecker O, Hsieh YH, Rothman RE, Kelen GD, Quinn TC. 2016. Parallel declines in HIV and hepatitis C virus prevalence, but not in herpes simplex virus type 2 infection: a 10-year, serial cross-sectional study in an inner-city emergency department. J Clin Virol 80:93–97.

Information & Contributors

Information

Published In

cover image Journal of Clinical Microbiology
Journal of Clinical Microbiology
Volume 59Number 718 June 2021
eLocator: 10.1128/jcm.00837-21
Editor: Yi-Wei Tang, Cepheid

History

Received: 7 April 2021
Returned for modification: 17 April 2021
Accepted: 23 April 2021
Accepted manuscript posted online: 26 April 2021
Published online: 18 June 2021

Permissions

Request permissions for this article.

Keywords

  1. lateral flow antibody assay
  2. SARS-CoV-2
  3. Uganda
  4. assay performance

Contributors

Authors

Owen R. Baker
Division of Intramural Research, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Baltimore, Maryland, USA
M. Kate Grabowski
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
Rakai Health Sciences Program, Kalisizo, Uganda
Ronald M. Galiwango
Rakai Health Sciences Program, Kalisizo, Uganda
Aminah Nalumansi
Uganda Virus Research Institute, Entebbe, Uganda
Jennifer Serwanga
Uganda Virus Research Institute, Entebbe, Uganda
Medical Research Council, Uganda Virus Research Institute and London School of Hygiene and Tropical Medicine Uganda Research Unit, Entebbe, Uganda
William Clarke
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
Yu-Hsiang Hsieh
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
Richard E. Rothman
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
Reinaldo E. Fernandez
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
David Serwadda
Rakai Health Sciences Program, Kalisizo, Uganda
Makerere University School of Public Health, Kampala, Uganda
Joseph Kagaayi
Rakai Health Sciences Program, Kalisizo, Uganda
Tom Lutalo
Rakai Health Sciences Program, Kalisizo, Uganda
Uganda Virus Research Institute, Entebbe, Uganda
Steven J. Reynolds
Division of Intramural Research, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Baltimore, Maryland, USA
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
Rakai Health Sciences Program, Kalisizo, Uganda
Pontiano Kaleebu
Uganda Virus Research Institute, Entebbe, Uganda
Medical Research Council, Uganda Virus Research Institute and London School of Hygiene and Tropical Medicine Uganda Research Unit, Entebbe, Uganda
Thomas C. Quinn
Division of Intramural Research, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Baltimore, Maryland, USA
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
Division of Intramural Research, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Baltimore, Maryland, USA
Johns Hopkins University School of Medicine, Baltimore, Maryland, USA

Editor

Yi-Wei Tang
Editor
Cepheid

Metrics & Citations

Metrics

Note: There is a 3- to 4-day delay in article usage, so article usage will not appear immediately after publication.

Citation counts come from the Crossref Cited by service.

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

View Options

Figures and Media

Figures

Media

Tables

Share

Share

Share the article link

Share with email

Email a colleague

Share on social media

American Society for Microbiology ("ASM") is committed to maintaining your confidence and trust with respect to the information we collect from you on websites owned and operated by ASM ("ASM Web Sites") and other sources. This Privacy Policy sets forth the information we collect about you, how we use this information and the choices you have about how we use such information.
FIND OUT MORE about the privacy policy