ABSTRACT

Serological assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) are needed to support clinical diagnosis and epidemiological investigations. Recently, assays for large-scale detection of total antibodies (Ab), immunoglobulin G (IgG), and IgM against SARS-CoV-2 antigens have been developed, but there are limited data on the diagnostic accuracy of these assays. This study was a Danish national collaboration and evaluated 15 commercial and one in-house anti-SARS-CoV-2 assays in 16 laboratories. Sensitivity was evaluated using 150 samples from individuals with asymptomatic, mild, or moderate COVID-19, nonhospitalized or hospitalized, confirmed by nucleic acid amplification tests (NAAT); samples were collected 13 to 73 days either from symptom onset or from positive NAAT (patients without symptoms). Specificity and cross-reactivity were evaluated in samples collected prior to the SARS-CoV-2 epidemic from >586 blood donors and patients with autoimmune diseases, cytomegalovirus or Epstein-Barr virus infections, and acute viral infections. A specificity of ≥99% was achieved by all total-Ab and IgG assays except one, DiaSorin Liaison XL IgG (97.2%). Sensitivities in descending order were Wantai ELISA total Ab (96.7%), CUH-NOVO in-house ELISA total Ab (96.0%), Ortho Vitros total Ab (95.3%), YHLO iFlash IgG (94.0%), Ortho Vitros IgG (93.3%), Siemens Atellica total Ab (93.2%), Roche Elecsys total Ab (92.7%), Abbott Architect IgG (90.0%), Abbott Alinity IgG (median 88.0%), DiaSorin Liaison XL IgG (median 84.6%), Siemens Vista total Ab (81.0%), Euroimmun/ELISA IgG (78.0%), and Snibe Maglumi IgG (median 78.0%). However, confidence intervals overlapped for several assays. The IgM results were variable, with the Wantai IgM ELISA showing the highest sensitivity (82.7%) and specificity (99%). The rate of seropositivity increased with time from symptom onset and symptom severity.

INTRODUCTION

In late December 2019, the World Health Organization (WHO) was notified of a cluster of cases of pneumonia in Wuhan City, China. The virus responsible was isolated in the first week of January 2020, and its genome was shared a week later. Phylogenetic analysis showed that it was a novel coronavirus, designated initially as 2019 novel coronavirus (2019-nCoV) and later as severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2). SARS‐CoV‐2 quickly spread worldwide, and the WHO declared coronavirus disease 2019 (COVID-19) a pandemic on 11 March 2020 (1).
In the following months, several hundred assays for detecting SARS-CoV-2 emerged. Different versions of nucleic acid amplification tests (NAATs) for naso-/oropharyngeal swabs or washes and lower respiratory tract specimens are essential in diagnosis of COVID-19 (2). However, assays for detecting antibodies produced as part of the humoral immune response to SARS-CoV-2 infection have emerged (3). These assays show that 1 week after the first symptoms, 30% of patients with COVID-19 have seroconverted, increasing to 70% after the second week and to above 90% by the third week (4). Accordingly, serological assays measuring total antibodies (Ab), immunoglobulin G (IgG), or IgM against antigens of SARS‐CoV‐2 have been used for supporting a diagnosis of COVID-19, for monitoring the epidemic, and for screening recovered COVID-19 patients for use in convalescent plasma therapy (5). Currently, the numerous serological assays have been validated on a limited number of samples and have at best been approved for emergency use after only a few days of evaluation. Several serological assays, especially the lateral-flow point-of-care tests, have a suboptimal performance with a low sensitivity and are not recommended for diagnostic use or even for population monitoring (68). Recently, several manufacturers of larger platforms have released serological assays useful for mass testing, but few studies have compared these assays directly (9). This comparison is needed for the commutability of the test results and the scientific data. Here, we present a national evaluation of 16 serological SARS-CoV-2 immunoassays across 16 laboratories in Denmark.

MATERIALS AND METHODS

Case panel samples for determination of clinical sensitivity.

The case panel samples tested in all assays (n = 150) were obtained from convalescent patients in the Capital Region of Denmark with a confirmed SARS-CoV-2 NAAT result that were identified in the Danish Microbiology Database from February 2020 to April 2020 (10). A total of 3,692 individuals were contacted via public secure mail and 639 persons responded. Serum samples and EDTA samples were obtained from respondents from 3 to 11 May 2020. For this study, only the first 150 consecutively collected serum samples from 3 May were chosen without any further selection and sent to all participating laboratories. Epidemiologic and clinical data were self-reported in an electronic questionnaire completed on the day of blood sampling.

Archived samples for determination of clinical specificity.

Archived plasma samples from regional pre-COVID-19 blood donations drawn during the influenza seasons of 2017–2018 and 2018–2019 were tested. The numbers of tested samples were >586 for the total-Ab and IgG assays and >400 for the IgM assays. Different sample sets and sample sizes were used across regions, with minor overlap in some cases. The specificities were calculated by combining the data from all sites that validated the same assay.

Archived samples for determination of cross-reactivity.

For all assays, cross-reactivity was investigated by testing samples from patients with unspecified autoimmune diseases (n = 10 to 131). Due to challenges with available amounts of sample material, 10 samples were pooled and tested across all assays. The nonpooled samples were tested in selected assays. Additionally, for all assays, archived local samples from patients with acute infections of cytomegalovirus (CMV) or Epstein-Barr virus (EBV) or other acute viral respiratory infections (respiratory syncytial virus, influenza A and B viruses, and adenovirus) based on positive IgM serology were tested (n = 10 to 37). Different sample sets were used across assays and laboratories. All samples were obtained prior to January 2020, before the first COVID-19 case in Denmark.

Immunoassay platforms.

The diagnostic accuracies of commercial immunoassays for the detection of anti-SARS-CoV-2 total Ab, anti-SARS-CoV-2 IgG, and anti-SARS-CoV-2 IgM were tested on the appropriate platforms by experienced laboratory technicians following the manufacturers’ protocols with the cutoff values suggested by the manufacturers (Table 1).
TABLE 1
TABLE 1 Serological immunoassays testeda
ManufacturerLaboratory no.AnalyzerCatalog no.Lot no.Manufacturer’s cutoffAntigenAb typeMethod
Roche Diagnostics, Mannheim,
Germany
5+6Elecsys09175431190495040, 490258Negative, <1.0 COI;
positive, ≥1.0 COI
rNTotalSandwich CLIA
Siemens Healthcare, Tarrytown,
NY, USA
4Atellica IM11206711C2T01, C2T02Negative, <1.0 COI;
positive, ≥1.0 COI
rS1 RBDTotalSandwich CLIA
7Dimension
Vista 500
K741420148BANegative, <1000 QU;
positive, ≥1,000 QU
rS1 RBDTotalSandwich CLIA
Abbott, Abbott Park, IL, USA10+11+12Alinity06R9017065FN00,
17069FN00
Negative, <1.4 (S/C);
positive, ≥1.4 (S/C)
rNIgGSandwich CMIA
9ArchitectR68616253FN00Negative, <1.4 (S/C);
positive, ≥1.4 (S/C)
rNIgGSandwich CMIA
Snibe, Shenzhen, China4+13+14Maglumi 4000+130219015M272200501Negative, <1.0;
positive, ≥1.0
r2019-nCoV,
nonspecified
IgGIndirect CLIA
4+13+14Maglumi 800130219016M271200401Negative, <1.0;
positive, ≥1.0
r2019-nCoV,
nonspecified
IgMCapture CLIA
Ortho Clinical Diagnostics,
Pencoed, UK
3Vitros 3600619 99220012, 0021, 0151Negative, <1.0 (S/CO);
positive, ≥1.0 (S/CO)
rS1 RBDTotalSandwich CLIA
3Vitros 3600619 99190100Negative, <1.0 (S/CO);
positive, ≥1.0 (S/CO)
rS1 RBDIgGSandwich CLIA
Wantai, Beijing, China1ELISAWS-1096NCOA20200301,
NCOA20200401
Negative, <1.1 (S/CO);
positive, ≥1.1 (S/CO)
rS RBDTotalDouble antigen sandwich
ELISA
10ELISAWS-1196NCOM20200301Negative, <1.1 (S/CO);
positive, ≥1.1 (S/CO)
rS RBDIgMTwo-step solid-phase
antibody capture ELISA
DiaSorin, Saluggia, Italy13+14+ 15+16Liaison XL311450354009, 354010Negative, <12 AU/ml;
equivocal, 12–15 AU/ml;
positive, ≥15 AU/ml
rS1, rS2 RBDIgGIndirect CLIA
Euroimmun, Lübeck,
Germany
9+10ELISAEI 2606-9601 GE200408AO,
E200420AW,
E200423AF,
E200317BP,
E200416AE
Negative, <0.8; borderline,
≥0.8 to <1.1; positive, ≥1.1
rS1 RBDIgGSolid phase, Enzyme
immunoassay
Copenhagen University
Hospital and Novo
Nordisk A/S, Denmark
2ELISAIn houseNegative, <9.4 (S/CON);
positive ≥9.4 (S/CON)
rS RBDTotalDouble antigen sandwich
ELISA
YHLO Biotechnology, Shenzhen,
China
8iFlash 180020200301, 20200212Negative, <10 AU/ml;
positive, ≥10 AU/ml
rSIgGIndirect CLIA
8iFlash 180020200301, 20200212Negative, <8 AU/ml;
positive, ≥8 AU/ml
rSIgMIndirect CLIA
a
COI, cutoff index; S/C, stored-calibrator index; S/CO, sample-to-cutoff ratio; QU, qualitative units; S/CON, sample-to-control ratio; CLIA, chemiluminescence immunoassay; FIA, fluorescence immunoassay; CMIA, chemiluminescence microparticle immunoassay; r, recombinant; S, spike antigen; RBD, receptor binding domain; N, nucleocapsid antigen; IgG, immunoglobulin G; IgM, immunoglobulin M.

ELISA.

The commercial enzyme-linked immunosorbent assays (ELISAs) for anti-SARS-CoV-2 total-Ab, IgG and IgM detection were performed on open-platform analyzers or manually according to the manufacturers’ instructions. The Euroimmun SARS-CoV-2 IgG assay was performed on an Analyzer I (Euroimmun AG, Lübeck, Germany), Quanta-Lyser 160 (INOVA Diagnostics, San Diego, CA, USA), or Evolis (Bio-Rad, Hercules, CA, USA). The Wantai SARS CoV-2 total-Ab and IgM assays were performed manually and measured using a Tecan Sunrise ELISA reader (Männedorf, Switzerland) at 450 nm with reference at 620 nm.
One in-house ELISA detecting total Ab was tested in this evaluation. Briefly, the CUH-NOVO SARS-CoV-2 total-Ab ELISA (a noncommercial assay produced in a collaboration between Copenhagen University Hospital and Novo Nordisk A/S, Denmark) is based on a recombinant receptor-binding domain (RBD) of the SARS-CoV-2 spike protein used for both coating and detection (11). Briefly, the samples were diluted 1:100 in phosphate-buffered saline with Tween 20 (PBS-T) in 96-well plates coated with RBD. Total Ab was detected using horseradish peroxidase (HRP)-conjugated streptavidin diluted in PBS-T mixed with biotin-labeled RBD. TMB One was used as a substrate. The reaction was stopped with 0.3 M H2SO4, and the optical density of the samples was measured at 450 to 620 nm. The CUH-NOVO SARS-CoV-2 total-Ab ELISA used a semiautomated setup, and the results were based on signal-to-noise ratios between the samples of interest and the negative quality control. The cutoff value was calculated based on receiver operating characteristic (ROC) analysis by prioritizing the specificity.
The manufacturers of the Euroimmun ELISA and the DiaSorin Liaison XL assay have defined gray zone/borderline results. As high specificity was prioritized at the cost of some sensitivity, borderline results were interpreted as negative (see Table S3 in the supplemental material).
Some assays were evaluated in more than one laboratory.

Statistics.

Data handling, graphics, and statistics were performed using the R statistical software (12). The parameters of diagnostic accuracy and the plots were determined using the mada package (13). For calculation of the 95% confidence intervals for the sensitivity and specificity, the default “Wilson” option was chosen. For plotting of bivariate confidence regions in the ROC space, a continuity correction of 0.5 was applied, as there were cells with zero counts.

Performance criteria.

We defined acceptance criteria for the diagnostic accuracy of the assays depending on immunoglobulin type and intended use.

Ethics statement.

The study of samples from patients with former SARS-CoV-2 infection for validation of serological SARS-CoV-2 assays was approved by the Regional Committee on Health Research Ethics for the Capital Region of Denmark (H-20028627) and was conducted in accordance with this approval. All blood donors are routinely asked at every blood donation for consent for future use of archived samples in the validation of new methods and assay investigations as quality control projects.

RESULTS

Of 150 patient samples, epidemiologic and clinical data were available for 149 patients; for the patient characteristics, refer to Table 2. Most patients were categorized with clinically mild to moderate symptoms (n = 112), and only 31 patients had been admitted to hospital. Time from symptom onset (TSO) was >21 days for 120 case panel samples, of which 79 were collected >6 weeks after symptom onset. The shortest TSO was 13 days, and the TSO was unknown for 9 patients.
TABLE 2
TABLE 2 Case panel patient characteristics
CharacteristicaValue (%)b
Sex
 Male52 (34.9)
 Female97 (65.1)
Median age (IQR) (yr)54 (43–64)
Age range (yr)18–83
TSO (days)
 0–70 (0.0)
 >7–147 (4.7)
 >14–2113 (8.7)
 >21–4249 (32.7)
 >4271 (47.3)
 NA9 (6.0)
Time from positive SARS–CoV–2 PCR (days)
 0–71 (0.7)
 >7–1415 (10.0)
 >14–2122 (14.6)
 >21–4290 (60.0)
 >4221 (47.3)
Symptom severity
 No symptoms6 (4.0)
 Mild (at home, well)37 (24.8)
 Moderate (home, bedridden)75 (50.3)
 Severe (hospitalized)2 (1.3)
 Critical (assisted ventilation)29 (19.5)
Total149 (100)
a
IQR, interquartile range; NA, not available.
b
Values are numbers of patients unless otherwise specified.

Data on the detection of anti-SARS-CoV-2 antibodies by each assay.

The results of the tests, i.e., true positives (TP), false negatives (FN), false positives (FP), and true negatives (TN), as well as the calculated sensitivity and specificity of each assay, are presented in Table 3 in descending order of sensitivity.
TABLE 3
TABLE 3 Anti-SARS-CoV-2 antibody assays with resultsa
Manufacturer/platformAssayLaboratorybNo. of:No. of FP cross-reactive samples/total% sensitivity (95% CI)% specificity (95% CI)% sensitivity in 120 samples with TSO >21 daysc
TPFNFPTN
AutoEBV, CMV
Total-Ab assays
 Wantai ELISA11+10145536560/650/3796.7 (92.4–98.6)99.5 (98.7–99.8)97.6
 In-house CUH-NOVO ELISA22144636170/500/2596.0 (91.5–98.2)99.5 (98.6–99.8)95.9
 Ortho CD Vitros33143706050/500/2095.3 (90.7–97.7)100.0 (99.4–100)95.9
 Siemens Atellica441381035930/500/2593.2 (87.9–96.7)99.5 (98.5–99.8)96.7
 Roche Elecsys55139112216NDND92.7 (87.3–95.9)99.1d (96.7–99.7)95.9
61381206100/600/2592.0 (86.5–95.4)100.0 (99.4–100)95.9
 Siemens Vista671192805960/100/2581.0 (73.7–87.0)100.0 (99.4–100)81
 
IgG assays
 YHLO iFlash78141945821/500/2594.0 (89.0–96.8)99.3 (98.3–99.7)95.9
 Ortho CD Vitros831401006000/500/2593.3 (88.2–96.3)100.0 (99.4–100)95.9
 Abbott Architect991351536000/251/3290.0 (84.2–93.8)99.5 (98.5–99.8)93.5
 Abbott Alinity10101341645960/500/2589.3 (83.3–93.8)99.3 (98.3–99.7)93.5
11132180/50ND88.0 (81.8–92.3)91.9
12132180/53ND88.0 (81.8–92.3)91.9
 Euroimmun ELISA119+101173355940/500/3578.0 (70.7–83.9)99.2 (98.1–99.6)82.9
 Snibe Maglumi12131163291,1640/500/1078.4 (71.1–84.2)99.2 (98.5–99.6)82.8
14117330/50ND78.0 (70.5–84.4)82.9
411337NDND75.3 (67.6–82.0)81.0
 DiaSorin Liaison XL131412822391,3491/600/2585.3 (78.8–90.1)97.2 (96.2–97.9)89.4
15127231/600/2584.7 (77.9–90.0)88.6
13125232/500/1084.5 (77.6–89.9)87.7
16123271/502/2582.0 (74.9–87.8)87.0
 
IgM assays
 Wantai ELISA14101242643960/530/2582.7 (75.8–87.9)99.0 (97.5–99.6)
 YHLO iFlash158638725830/502/2542.0 (34.4–50.0)99.7 (98.8–99.9)
 Snibe Maglumi16146387441,1401/50ND42 (34.4–50.0)96.3 (95.0–97.3) 
13451030/500/1030.4 (23.1–38.5)
439109NDND26.4 (19.5–34.2)
a
IgG, Immunoglobulin G; IgM, Immunoglobulin M; total-Ab, total antibodies; TP, true positive; FN, false negative; FP, false positive; TN, true negative; Auto, pre-COVID-19 samples from patients with autoimmune diseases; EBV and CMV, preCOVID-19 samples from patients with acute Epstein-Barr virus or cytomegalovirus or other acute viral infections; ND, not done.
b
The key to each laboratory is presented in Appendix S1 (Table S1) in the supplemental material.
c
The mean size of the 95% confidence intervals for the sensitivities of the samples collected 21 days or later after symptom onset is 8% for total-Ab assays and 11% for IgG assays.
d
Patient samples from hospitalized patients from before January 2020.
All total-Ab assays performed with high specificities (≥99%). Two total-Ab ELISAs and the Ortho Vitros total-Ab assay performed with sensitivities of ≥95%, while the Siemens Atellica and Roche Elecsys assays performed with sensitivities of ≥92%. One assay, the Siemens Vista assay, performed with a sensitivity of only 81%.
Of the IgG assays, all but the DiaSorin Liaison XL IgG assay performed with specificities of ≥99%. Three assays (the YHLO iFlash IgG, the Ortho Clinical Diagnostics (Ortho CD) Vitros IgG, and the Abbott Architect IgG assays) showed sensitivities of ≥90%.
The sensitivity improved in all total-Ab and IgG assays if the analyses were restricted to samples collected >21 days after symptom onset (Table 3).
Regarding the IgM assays, the Wantai IgM ELISA demonstrated a higher sensitivity than the other IgM assays, with a specificity of 99.0% and no cross-reactivity (Fig. 1 and Table 3). The sensitivity of the YHLO iFlash IgM assay was 42%, and cross-reactivity was detected in two of 25 samples from pre-COVID-19 patients with either acute CMV or EBV infections, whereas the Snibe Maglumi IgM assay performed with a specificity of 96.3% and a sensitivity of 26.4 to 42% (Fig. 1).
FIG 1
FIG 1 Summary ROC plot of sensitivity and false-positive rate with elliptic 95% bivariate confidence regions corresponding to the data in Table 3 for assays with total Ig, IgG, and IgM, respectively. For the IgG assays where data were available from more than one laboratory, the median result was chosen for the COVID-19 cases, and for the prepandemic blood donors, the total of all the samples was used, as these were from different persons. The vertical broken line at a false-positive rate of 0.01 corresponds to a 99% specificity. The y axis for IgM has a different scale, from 20% sensitivity instead of 70%.
Quantitative ROC analysis is provided in Appendix S2 in the supplemental material. The total-Ab assays all had discriminatory ability, with areas under the curves (AUC) of ≥97%, whereas more variation was seen within the IgG assays, with AUCs ranging from 91.9% to 99.3%. The IgM assays had a larger variation in AUC, ranging from 75.7% to 98%.
Pairwise comparison of identical samples from 150 COVID-19 patients showed a high degree of variation between assays but nearly identical results when the same assay was applied in different laboratories. The details of this pairwise comparison are provided in Appendix S3 in the supplemental material.

Data on the detection of anti-SARS-CoV-2 antibodies according to TSO and disease severity.

The largest variation in true-positive samples between the assays for each immunoglobulin (Ig) type category was shown among samples with a TSO of ≤21 days (Fig. 2). In addition, increasing rates of seropositive results were found with the severity of symptoms in all assays (Fig. 2).
FIG 2
FIG 2 Antibody development for total-Ab (IgT), IgG, and IgM as a function of days from symptom onset in 3-week periods and severity of symptoms. Assays are color coded. Multiple lines with the same color appear if the same assay was performed in different laboratories, to show the (small) interlaboratory variation.
In four samples SARS-CoV-2 antibodies were not detected by any of the assays. Of these, three patients had mild clinical symptoms and one patient had moderate symptoms, with the TSO varying between 17 and 73 days (data not shown).

DISCUSSION

In the United States, the Food and Drug Administration requires a minimum sensitivity of 90% and a specificity of 95% for emergency use authorization of serologic anti-SARS-CoV-2 assays (14). In the United Kingdom, the Medicines and Healthcare Products Regulatory Agency has defined a “target product profile” for enzyme immunoassays detecting antibodies against SARS-CoV-2 (15), defining an acceptable sensitivity and specificity of ≥98% for anti-SARS-CoV-2-immunoassays among patients with a history of SARS-CoV-2 ≥20 days after symptom onset.
In a low-seroprevalence setting, the specificity of the test is the most important concern and must be high. For this reason, irrespective of specific clinical indication for the use of the anti-SARS-CoV-2 total-Ab and anti-SARS-CoV-2 IgG assays, we defined the specificity acceptance criterion as ≥99%. For the SARS-CoV-2 total-Ab assays, the defined acceptable sensitivity for diagnostic use was ≥92%, with an optimal sensitivity of ≥95%. For the anti-SARS-CoV-2 IgG assays, the defined acceptable sensitivity was ≥90%. For epidemiologic surveys, the acceptable sensitivity was defined as ≥80% for both the anti-SARS-CoV-2 total-Ab and the anti-SARS-CoV-2 IgG assays, as statistical adjustments for low sensitivity can be performed. No acceptance criteria for the diagnostic accuracy of the anti-SARS-CoV-2 IgM assays were defined, as most of the samples in the case panel were collected >21 days after symptom onset.
This study demonstrated that diagnostic accuracy was higher in the group of the SARS-CoV-2 total-Ab assays than the group of the SARS-CoV-2 IgG assays. All total-Ab assays but one exhibited acceptable performance for diagnostic use with regard to both specificity and sensitivity. The SARS-CoV-2 IgG assays demonstrated a larger variation in sensitivities, and only three assays showed acceptable sensitivity for diagnostic use, whereas one assay did not meet the defined acceptance criteria for specificity. All total-Ab and IgG assays showed higher seropositive rates in samples from patients with a TSO of >21 days, and seropositive rates increased with symptom severity in all assays across all Ig types.

Total-Ab assays.

Nominally, the accuracy was superior in the Wantai total-Ab assay, the in-house CUH-NOVO total-Ab ELISA, and the Ortho CD Vitros total-Ab assay, with optimal sensitivity for diagnostic use according to our criteria. The Roche Elecsys total-Ab and the Siemens Atellica total-Ab assays also performed with acceptable sensitivity for diagnostic use, and the confidence intervals for sensitivity and specificity overlapped the predefined acceptance criteria for optimal performance for the mentioned assays. The true values for sensitivity and specificity might therefore fulfill the defined criteria for optimal performance. The poor sensitivity of the Siemens Vista assay was seemingly due to the manufacturer’s setting with a suboptimal assigned cutoff value, as the ROC analysis showed an AUC similar to those of the other assays (see Fig. S2). Thus, adjusting the cutoff used in the Siemens Vista assay appears relevant for improving diagnostic performance. However, for this study, we used the cutoffs specified by the manufacturers.

IgG assays.

Three assays—YHLO iFlash IgG, Ortho CD Vitros IgG, and Abbott Architect IgG—showed acceptable sensitivity for diagnostic use per our predefined acceptance criteria. The confidence intervals for sensitivity in the Abbott Alinity IgG assay included the predefined cutoffs for acceptable performance, and it is possible that the true sensitivity of the assay is acceptable. Regarding specificity, only the DiaSorin Liaison XL IgG assay did not meet the defined criteria.

IgM assays.

More than half of the samples in the case panel had a TSO of >6 weeks and were not optimal for the evaluation of the sensitivity of the IgM assays. However, among the IgM assays, the Wantai IgM assay stood out with a relatively high sensitivity and specificity.

Antigen specificity.

In three of the evaluated assays, a recombinant nucleocapsid antigen (rN) is used in the immunoassay, while in eight assays, a recombinant spike antigen (rS) of the RBD is used; two assays did not specify the protein(s) used as the capturing antigen in the assay, and two assays (the YHLO IgG and IgM) use both rN and rS. Our study does not suggest that the chosen antigen (N versus S RBD) affects the assay performance per se. Instead, the differences in performance seem to be associated with overall assay design rather than the choice of antigen. This parallels the observations made by Haveri et al., who demonstrated the appearance of neutralizing antibodies against both N and S proteins simultaneously (16).

Cross-reactivity.

A potential limitation for the use of immunoassays can be interference due to cross-reactivity in individuals with autoimmune disease, infections with other respiratory viruses, or acute infections with EBV or CMV. We tested this by using samples from patients with autoimmune diseases, samples from pre-COVID-19 patients with viral infections, and pre-COVID-19 samples from blood donors who very likely had been exposed to various respiratory viruses, including non-SARS-CoV-2 coronaviruses. This did not seem to be an issue in most assays, except for the DiaSorin Liaison XL IgG and YHLO iFlash IgM assays, which showed interference in some samples.
In our study, the sensitivities calculated from the case samples with a known TSO of >21 days did not reach 98%, as defined by the UK authorities, in any assay evaluated (Table 3). However, we prioritized high diagnostic specificity (≥99%) as our main criterion, since Denmark has a low anti-SARS-CoV-2 seroprevalence so far. For example, in the Danish population with a SARS-CoV-2 Ab prevalence of only 1.9% (April and May 2020) (17), an assay specificity of 97.2% (DiaSorin Liaison XL) would lead to a low positive predictive value (PPV) of 34%, while a test with a higher specificity of 99.5% (Abbott Architect) would yield a PPV of 76%. The consequences of a low specificity are less pronounced in a higher-prevalence setting; for example, with an antibody prevalence of 22%, as reported in parts of Iran (18), a low-specificity test like the DiaSorin assay will have a PPV of 89% and a more specific test such as the Abbott Architect a PPV of 98%. A recent evaluation of the DiaSorin Liaison XL assay did not include a large specificity panel of donor samples, but it also showed cross-reactivity in some samples from patients with rheumatoid factors or positivity for antinuclear antibodies, substantiating unspecific reactions in this assay (19). Interestingly, many manufacturers use a TSO of ≥14 days, in contrast to >21 days, as a cutoff for optimal sensitivity, indicating a need for international consensus on which TSO to test for optimal sensitivity (20).
A similar national validation study (21), with a large sample size, was performed in the United Kingdom; that study compared five immunoassays, of which four (DiaSorin Liaison XL IgG, Abbott Architect IgG, Roche Elecsys, and Siemens Atellica) were included in the present study. Generally, the study from the UK finds a higher sensitivity and specificity for all assays, including DiaSorin Liaison XL IgG, which (nearly) met the 98% sensitivity and specificity required by the UK authorities (15). This difference may be explained by the fact that the UK study included samples obtained at least 20 days after symptom onset. Based on our results, it is probably not realistic to expect to achieve a 98% sensitivity when a large proportion of milder cases are included in the cohort.
Generally, serological testing had a low sensitivity when carried out less than 3 weeks from symptom onset and in patients who were asymptomatic or had mild disease at home but were not bedridden. If patients had been bedridden or hospitalized, the sensitivity of serological testing of samples from convalescent patients was >90% for most of the total-Ab or IgG assays included in the study (Fig. 2). The large difference in sensitivity according to disease severity was not explained by the difference in time of testing, as two large groups of nonhospitalized symptomatic patients had similar median TSO near 40 days and a range of distribution from symptom onset to blood sampling (see Appendix S1 and Fig. S1).

Nonreactive cases.

We found four cases without detected anti-SARS-CoV-2 antibodies in any assay. This finding could possibly be explained by early-stage infection, mild disease, transient antibody response only, antibodies not produced or produced at nondetectable levels, late or slow antibody development, or false-positive NAATs.
Early-stage infection was not the case in these four patients, who had a TSO of 17 to 73 days, whereas the largest variation in sensitivity performance between the anti-SARS-CoV-2 assays was seen in the samples with a TSO of ≤21 days. The TSO among the COVID-19 case samples indeed determined the absolute sensitivity values obtained, as the median seroconversion time is reportedly 11 days (interquartile range of 7.3 to 14.0 days) after onset of symptoms (2225).
Of the four patients, three had mild disease and one moderate disease. A combination of mild COVID-19 symptoms and the collection of blood samples in the late convalescent stage might explain the nondetectable antibodies. As we showed, most anti-SARS-CoV-2 total-Ab and IgG assays had a higher rate of seropositivity among hospitalized patients than nonhospitalized patients. Previous studies have also shown that anti-SARS-CoV-2 titers correlate with the severity of COVID-19 among hospitalized patients (2628) and that there is a time-dependent decline in antibody titers for anti-SARS-CoV-2 Ab in general, including neutralizing antibodies (17, 29).
The false-positive rate for NAATs is estimated to be between 0.8 and 4% in the United Kingdom (30), which could explain the four antibody-negative patients among the 150 NAAT-positive patients in the panel.
Our data provide some interesting preliminary observations regarding the antibody response in general. First, most individuals (approximately 97%) seem to develop some degree of antibody response. Second, this response seems to peak in samples collected approximately 3 weeks after TSO. Third, the response seems positively correlated with disease severity.
It is important to point out that while our study compares the ability of antibody assays to identify individuals who have had NAAT-confirmed COVID-19, it does not compare the ability to identify individuals who are protected against reinfection with SARS-CoV-2. Identification of those who are protected against reinfection, at least for a certain period, would be an important aspect of an assay, but it is too early in the epidemic to make this kind of comparison on a large scale. A recent study from GeurtsvanKessel et al. (31) reported cutoff values in the Wantai total-Ab assay indicating detectable levels of neutralizing antibodies. Those authors suggest this as a tool for the detection of neutralizing antibodies, though the clinical utility of this remains unclear.
Our study has several strengths and limitations. A major strength is that the case panel used for sensitivity across all 16 assays included 150 samples from the same patients, one of the largest panels investigated to date. Additionally, and in contrast to most previous evaluations of serological SARS-CoV-2 assays, this case panel was obtained largely from patients who had milder symptoms of COVID-19, evaluating whether the assays could detect SARS-CoV-2 antibodies among the most common type of patient with SARS-CoV-2 infection. This is valuable knowledge in seroepidemiological investigations. The specificity was evaluated with a significant number of pre-COVID-19 blood donor samples, making this study very solid in terms of clinical accuracy and agreement between the assays investigated. Furthermore, we investigated cross-reactivity using samples from individuals with autoimmune disease or acute infections with EBV or CMV. For these tests of specificity, we could not use samples from the same individuals across all assays due to the small sample volumes. This could potentially introduce heterogeneity between assays in the specificity data. However, using ≥586 samples from healthy donors for each assay makes this a very large sample, and substantial differences in a small homogenous country with same standard operating procedures for Danish blood banks are unlikely. Even though we tried to assess the risk of interference by examining samples from patients with acute viral diseases known to be associated with increased levels of assay-interfering antibodies, these could be present in patients with other diseases, e.g., other common coronavirus infections or cancers. Thus, we could have underestimated the potential for interference.
In conclusion, this comparative study of 15 commercial and one in-house laboratory serological SARS-CoV-2 assays pinpoints differences in accuracy; most total-Ab and IgG assays, including assays with potential for high-throughput production in automated laboratories, reached predefined criteria for acceptable performance, especially in samples from cases with a TSO over 20 days. Additionally, the antibody response seemed to be strongest among patients with more severe disease. It could appear as if the use of emergency authorizations has led to release of suboptimal assays in some cases, and simple measures such as optimization of cutoff values could lead to major improvements in performance. Thus, it is possible that optimized versions of some assays may be released in the near future.

ACKNOWLEDGMENTS

We thank the laboratory technicians in all laboratories who performed the validation of assays in this evaluation for their thorough and effective laboratory performance. Additionally, we thank the head of DEKS (Danish Institute for External Quality Assurance for Laboratories in the Health Sector), Gitte Henriksen, for distributing the samples from patients with autoimmune diseases. We thank Charlotte Helgstrand for providing the antigen used in the in-house CUH-NOVO total-Ab ELISA. The plasmid used for synthesizing the SARS-CoV2 RBD polypeptide for that assay was constructed and kindly contributed by the International AIDS Vaccine Initiative (IAVI) by the responsible IAVI employee, Joseph Jardine, Scripps Institute, La Jolla, CA, USA. Some of the assays evaluated were purchased and some were provided for evaluation from the manufacturer without costs. We thank the manufacturers for technical advice and support.
The development of the CUH-NOVO SARS-CoV-2 total-Ab ELISA was financially supported by grants from the Carlsberg Foundation (CF20-0045) and the Novo Nordisk Foundation (205A0063505).
R. B. Dessau reports personal fees from a Roche Diagnostics advisory board meeting in 2018 outside this work. All other authors declare no competing interests.

Supplemental Material

File (jcm.02596-20-s0001.pdf)
ASM does not own the copyrights to Supplemental Material that may be linked to, or accessed through, an article. The authors have granted ASM a non-exclusive, world-wide license to publish the Supplemental Material files. Please contact the corresponding author directly for reuse.

REFERENCES

1.
Ghebreyesus T. 2020. WHO Director-General's opening remarks at the media briefing on COVID-19—11 March 2020.
2.
Rabi FA, Al Zoubi MS, Kasasbeh GA, Salameh DM, Al-Nasser AD. 2020. SARS-CoV-2 and coronavirus disease 2019: what we know so far. Pathogens 9:231.
3.
Prompetchara E, Ketloy C, Palaga T. 2020. Immune responses in COVID-19 and potential vaccines: lessons learned from SARS and MERS epidemic. Asian Pac J Allergy Immunol 38:1–9.
4.
Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, Adriano A, Beese S, Dretzke J, Ferrante di Ruffano L, Harris IM, Price MJ, Dittrich S, Emperador D, Hooft L, Leeflang MM, Van den Bruel A, Cochrane COVID-19 Diagnostic Test Accuracy Group. 2020. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 6:CD013652.
5.
Theel ES, Slev P, Wheeler S, Couturier MR, Wong SJ, Kadkhoda K. 2020. The role of antibody testing for SARS-CoV-2: is there one? J Clin Microbiol 58:e00797-20.
6.
Whitman JD, Hiatt J, Mowery CT, Shy BR, Yu R, Yamamoto TN, Rathore U, Goldgof GM, Whitty C, Woo JM, Gallman AE, Miller TE, Levine AG, Nguyen DN, Bapat SP, Balcerek J, Bylsma SA, Lyons AM, Li S, Wong AW, Gillis-Buck EM, Steinhart ZB, Lee Y, Apathy R, Lipke MJ, Smith JA, Zheng T, Boothby IC, Isaza E, Chan J, Acenas DD, II, Lee J, Macrae TA, Kyaw TS, Wu D, Ng DL, Gu W, York VA, Eskandarian HA, Callaway PC, Warrier L, Moreno ME, Levan J, Torres L, Farrington LA, Loudermilk RP, Koshal K, Zorn KC, Garcia-Beltran WF, Yang D, et al. 2020. Evaluation of SARS-CoV-2 serology assays reveals a range of test performance. Nat Biotechnol 38:1174–1183.
7.
Bastos ML, Tavaziva G, Abidi SK, Campbell JR, Haraoui L, Johnston JC, Lan Z, Law S, MacLean E, Trajman A, Menzies D, Benedetti A, Khan FA. 2020. Diagnostic accuracy of serological tests for covid-19: systematic review and meta-analysis. BMJ 370:m2516.
8.
Ria Lassaunière AF, Harboe ZB, Nielsen AC, Fomsgaard A, Krogfelt KA, Jørgensen CS. 2020. Evaluation of nine commercial SARS-CoV-2 immunoassays. medRxiv. https://www.medrxiv.org/content/10.1101/2020.04.09.20056325v1.
9.
Kohmer N, Westhaus S, Ruhl C, Ciesek S, Rabenau HF. 2020. Brief clinical evaluation of six high-throughput SARS-CoV-2 IgG antibody assays. J Clin Virol 129:104480.
10.
Voldstedlund M, Haarh M, Molbak K, MiBa Board of Representatives. 2014. The Danish Microbiology Database (MiBa) 2010 to 2013. Euro Surveill 19:20667.
11.
Hansen CB, Jarlhelt I, Pérez-Alós L, Hummelshøj Landsy L, Loftager M, Rosbjerg A, Helgstrand C, Bjelke JR, Egebjerg T, Jardine JG, Sværke Jørgensen C, Iversen K, Bayarri-Olmos R, Garred P, Skjoedt MO. 2020. SARS-CoV-2 antibody responses are correlated to disease severity in COVID-19 convalescent individuals. J Immunol.
12.
R Core Team. 2020. R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/.
13.
Doebbler P. 2020. mada: meta-analysis of diagnostic accuracy. CRAN.R-project.org/package=mada.
14.
Duong YT, Wright CG, Justman J. 2020. Antibody testing for coronavirus disease 2019: not ready for prime time. BMJ 370:m2655.
15.
Medicines and Healthcare Products Regulatory Agency. 2020. Target product profile: enzyme immunoassay (EIA) antibody tests to help determine if people have antibodies to SARS-CoV-2. https://www.gov.uk/government/publications/how-tests-and-testing-kits-for-coronavirus-covid-19-work/target-product-profile-enzyme-immunoassay-eia-antibody-tests-to-help-determine-if-people-have-antibodies-to-sars-cov-2.
16.
Haveri A, Smura T, Kuivanen S, Osterlund P, Hepojoki J, Ikonen N, Pitkapaasi M, Blomqvist S, Ronkko E, Kantele A, Strandin T, Kallio-Kokko H, Mannonen L, Lappalainen M, Broas M, Jiang M, Siira L, Salminen M, Puumalainen T, Sane J, Melin M, Vapalahti O, Savolainen-Kopra C. 2020. Serological and molecular findings during SARS-CoV-2 infection: the first case study in Finland, January to February 2020. Euro Surveill 25:2000266.
17.
Erikstrup C, Hother CE, Pedersen OBV, Mølbak K, Skov RL, Holm DK, Sækmose SG, Nilsson AC, Brooks PT, Boldsen JK, Mikkelsen C, Gybel-Brask M, Sørensen E, Dinh KM, Mikkelsen S, Møller BK, Haunstrup T, Harritshøj L, Jensen BA, Hjalgrim H, Lillevang ST, Ullum H. 2020. Estimation of SARS-CoV-2 infection fatality rate by real-time antibody screening of blood donors. Clin Infect Dis. Epub ahead of print.
18.
Shakiba M, Hashemi Nazari SS, Mehrabian F, Rezvani SM, Ghasempour Z, Heidarzadeh A. 2020. Seroprevalence of COVID-19 virus infection in Guilan province, Iran. medRxiv.
19.
Herroelen PH, Martens GA, De Smet D, Swaerts K, Decavele AS. 2020. Humoral immune response to SARS-CoV-2. Am J Clin Pathol 154:610–619.
20.
Caruana G, Croxatto A, Coste AT, Opota O, Lamoth F, Jaton K, Greub G. 2020. Diagnostic strategies for SARS-CoV-2 infection and interpretation of microbiological results. Clin Microbiol Infect 26:1178–1182.
21.
National S-C-SAEG. 2020. Performance characteristics of five immunoassays for SARS-CoV-2: a head-to-head benchmark comparison. Lancet Infect Dis 20:1390–1400.
22.
Long Q-X, Liu B-Z, Deng H-J, Wu G-C, Deng K, Chen Y-K, Liao P, Qiu J-F, Lin Y, Cai X-F, Wang D-Q, Hu Y, Ren J-H, Tang N, Xu Y-Y, Yu L-H, Mo Z, Gong F, Zhang X-L, Tian W-G, Hu L, Zhang X-X, Xiang J-L, Du H-X, Liu H-W, Lang C-H, Luo X-H, Wu S-B, Cui X-P, Zhou Z, Zhu M-M, Wang J, Xue C-J, Li X-F, Wang L, Li Z-J, Wang K, Niu C-C, Yang Q-J, Tang X-J, Zhang Y, Liu X-M, Li J-J, Zhang D-C, Zhang F, Liu P, Yuan J, Li Q, Hu J-L, Chen J, et al. 2020. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med 26:845–848.
23.
Zhao J, Yuan Q, Wang H, Liu W, Liao X, Su Y, Wang X, Yuan J, Li T, Li J, Qian S, Hong C, Wang F, Liu Y, Wang Z, He Q, Li Z, He B, Zhang T, Fu Y, Ge S, Liu L, Zhang J, Xia N, Zhang Z. 2020. Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019. Clin Infect Dis 71:2027–2034.
24.
Lou B, Li TD, Zheng SF, Su YY, Li ZY, Liu W, Yu F, Ge SX, Zou QD, Yuan Q, Lin S, Hong CM, Yao XY, Zhang XJ, Wu DH, Zhou GL, Hou WH, Li TT, Zhang YL, Zhang SY, Fan J, Zhang J, Xia NS, Chen Y. 2020. Serology characteristics of SARS-CoV-2 infection since exposure and post symptom onset. Eur Respir J 56:2000763.
25.
Huang AT, Garcia-Carreras B, Hitchings MDT, Yang B, Katzelnick LC, Rattigan SM, Borgert BA, Moreno CA, Solomon BD, Trimmer-Smith L, Etienne V, Rodriguez-Barraquer I, Lessler J, Salje H, Burke DS, Wesolowski A, Cummings DAT. 2020. A systematic review of antibody mediated immunity to coronaviruses: kinetics, correlates of protection, and association with severity. Nat Commun 11:4704.
26.
Qu J, Wu C, Li X, Zhang G, Jiang Z, Li X, Zhu Q, Liu L. 2020. Profile of immunoglobulin G and IgM antibodies against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Clin Infect Dis 71:2255–2258.
27.
Seow J, Graham C, Merrick B, Acors S, Pickering S, Steel KJA, Hemmings O, O'Byrne A, Kouphou N, Galao RP, Betancor G, Wilson HD, Signell AW, Winstone H, Kerridge C, Huettner I, Jimenez-Guardeño JM, Lista MJ, Temperton N, Snell LB, Bisnauthsing K, Moore A, Green A, Martinez L, Stokes B, Honey J, Izquierdo-Barras A, Arbane G, Patel A, Tan MKI, O'Connell L, O'Hara G, MacMahon E, Douthwaite S, Nebbia G, Batra R, Martinez-Nunez R, Shankar-Hari M, Edgeworth JD, Neil SJD, Malim MH, Doores KJ. 2020. Longitudinal observation and decline of neutralizing antibody responses in the three months following SARS-CoV-2 infection in humans. Nat Microbiol 5:1598–1607.
28.
Liu ZL, Liu Y, Wan LG, Xiang TX, Le AP, Liu P, Peiris M, Poon LLM, Zhang W. 2020. Antibody profiles in mild and severe cases of COVID-19. Clin Chem 66:1102–1104.
29.
Seow JGC, Merrick B, Acors S, Steel KJA, Hemmings O, O'Bryne A, Kouphou N, Pickering S, Galao R, Betancor G, Wilson HD, Signell AW, Winstone H, Kerridge C, Temperton N, Snell L, Bisnauthsing K, Moore A, Green A, Martinez L, Stokes B, Honey J, Izquierdo-Barras A, Arbane G, Patel A, OConnell L, O'Hara G, MacMahon E, Douthwaite S, Nebbia S, Batra R, Martinez-Nunez R, Edgeworth JD, Neil SJD, Malim M, Doores K. 2020. Longitudinal evaluation and decline of antibody responses in SARS-CoV-2 infection. medRxiv. https://doi.org/10.1101/2020.07.09.20148429.
30.
Surkova E, Nikolayevskyy V, Drobniewski F. 2020. False-positive COVID-19 results: hidden problems and costs. Lancet Respir Med 8:1167–1168.
31.
GeurtsvanKessel CH, Okba NMA, Igloi Z, Bogers S, Embregts CWE, Laksono BM, Leijten L, Rokx C, Rijnders B, Rahamat-Langendoen J, van den Akker JPC, van Kampen JJA, van der Eijk AA, van Binnendijk RS, Haagmans B, Koopmans M. 2020. An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment. Nat Commun 11:3436.

Information & Contributors

Information

Published In

cover image Journal of Clinical Microbiology
Journal of Clinical Microbiology
Volume 59Number 520 April 2021
eLocator: 10.1128/jcm.02596-20
Editor: Yi-Wei Tang, Cepheid

History

Received: 9 October 2020
Returned for modification: 16 November 2020
Accepted: 9 February 2021
Accepted manuscript posted online: 12 March 2021
Published online: 20 April 2021

Permissions

Request permissions for this article.

Keywords

  1. SARS-CoV-2 antibody test
  2. evaluation
  3. anti-SARS-CoV-2 serology assay

Contributors

Authors

Department of Clinical Immunology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Department of Clinical Immunology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Shoaib Afzal
Department of Clinical Biochemistry, Copenhagen University Hospital, Herlev and Gentofte Hospital, Herlev, Denmark
Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
Pia R. Kamstrup
Department of Clinical Biochemistry, Copenhagen University Hospital, Herlev and Gentofte Hospital, Herlev, Denmark
Charlotte S. Jørgensen
Department of Virus & Microbiological Special Diagnostics, Statens Serum Institut, Copenhagen, Denmark
Marianne Kragh Thomsen
Department of Clinical Microbiology, Aarhus University Hospital, Aarhus, Denmark
Linda Hilsted
Department of Clinical Biochemistry, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Lennart Friis-Hansen
Department of Clinical Biochemistry, Copenhagen University Hospital, Bispebjerg, and Frederiksberg Hospital, Copenhagen, Denmark
Pal B. Szecsi
Department of Clinical Biochemistry, Holbæk Hospital, Holbæk, Denmark
Lise Pedersen
Department of Clinical Biochemistry, Holbæk Hospital, Holbæk, Denmark
Lene Nielsen
Department of Clinical Microbiology, Copenhagen University Hospital, Herlev and Gentofte Hospital, Copenhagen, Denmark
Cecilie B. Hansen
Laboratory of Molecular Medicine, Department of Clinical Immunology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Peter Garred
Laboratory of Molecular Medicine, Department of Clinical Immunology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Department of Clinical Medicine, Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark
Trine-Line Korsholm
Department of Clinical Immunology, Aarhus University Hospital, Aarhus, Denmark
Susan Mikkelsen
Department of Clinical Immunology, Aarhus University Hospital, Aarhus, Denmark
Kirstine O. Nielsen
Department of Clinical Immunology, Aarhus University Hospital, Aarhus, Denmark
Bjarne K. Møller
Department of Clinical Immunology, Aarhus University Hospital, Aarhus, Denmark
Anne T. Hansen
Department of Clinical Immunology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Kasper K. Iversen
Department of Cardiology, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Emergency Medicine, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Pernille B. Nielsen
Department of Cardiology, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Emergency Medicine, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Rasmus B. Hasselbalch
Department of Cardiology, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Emergency Medicine, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Kamille Fogh
Department of Cardiology, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Emergency Medicine, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Jakob B. Norsk
Department of Cardiology, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Emergency Medicine, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Jonas Henrik Kristensen
Department of Cardiology, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Emergency Medicine, Herlev og Gentofte Hospital, University of Copenhagen, Herlev, Denmark
Department of Clinical Microbiology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Nikolai S. Kirkby
Department of Clinical Microbiology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Alex C. Y. Nielsen
Department of Clinical Microbiology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Lone H. Landsy
Department of Nonclinical and Clinical Assay Sciences in Global Discovery & Development Sciences, Novo Nordisk A/S, Måløv, Denmark
Mette Loftager
Department of Nonclinical and Clinical Assay Sciences in Global Discovery & Development Sciences, Novo Nordisk A/S, Måløv, Denmark
Dorte K. Holm
Department of Clinical Immunology, Odense University Hospital, Odense, Denmark
Anna C. Nilsson
Department of Clinical Immunology, Odense University Hospital, Odense, Denmark
Susanne G. Sækmose
Department of Clinical Immunology, Zealand University Hospital, Næstved Hospital, Næstved, Denmark
Birgitte Grum-Schwensen
Department of Clinical Immunology, Zealand University Hospital, Næstved Hospital, Næstved, Denmark
Bitten Aagaard
Department of Clinical Immunology, Aalborg University Hospital, Aalborg, Denmark
Thøger G. Jensen
Department of Clinical Microbiology, Odense University Hospital, Odense, Denmark
Research Unit for Clinical Microbiology, Odense University Hospital, University of Southern Denmark, Odense, Denmark
Dorte M. Nielsen
Department of Clinical Microbiology, Zealand University Hospital, Slagelse Hospital, Slagelse, Denmark
Henrik Ullum
Department of Clinical Immunology, Copenhagen University Hospital, Rigshospitalet, Copenhagen, Denmark
Department of Clinical Medicine, Faculty of Health Sciences, University of Copenhagen, Copenhagen, Denmark
Statens Serum Institut, Copenhagen, Denmark
Department of Clinical Microbiology, Zealand University Hospital, Slagelse Hospital, Slagelse, Denmark

Editor

Yi-Wei Tang
Editor
Cepheid

Metrics & Citations

Metrics

Note: There is a 3- to 4-day delay in article usage, so article usage will not appear immediately after publication.

Citation counts come from the Crossref Cited by service.

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

View Options

Figures and Media

Figures

Media

Tables

Share

Share

Share the article link

Share with email

Email a colleague

Share on social media

American Society for Microbiology ("ASM") is committed to maintaining your confidence and trust with respect to the information we collect from you on websites owned and operated by ASM ("ASM Web Sites") and other sources. This Privacy Policy sets forth the information we collect about you, how we use this information and the choices you have about how we use such information.
FIND OUT MORE about the privacy policy