Open access
Clinical Microbiology
Research Article
22 November 2024

Development of a metabolome-based respiratory infection prognostic during COVID-19 arrival

ABSTRACT

In a new respiratory virus pandemic, optimizing allocation of scarce medical resources becomes an urgent challenge. Infection prognosis takes on particular importance when allocating scarce antiviral antibodies and drugs, which are most effective when administered before the onset of severe disease. During arrival of the COVID-19 pandemic to the United States in 2020, we conducted a prognostic biomarker discovery and validation effort based upon metabolomic profiling with a liquid-chromatography-mass spectrometer (LC-MS) type used clinically for rapid toxicology. We obtained urine specimens from 163 patients presenting for evaluation. We obtained LC-MS profiles in the initial cohort and used machine learning methods to define a simplified urine metabolomic signature associated with respiratory failure or death by 90 days. This signature was composed of three metabotypes linked to intestinal microbiome metabolism and anticonvulsant use, with a receiver-operator characteristic area under the curve (ROC AUC) of 89.4%. Blinded application of this signature to the subsequent validation cohort yielded a ROC AUC of 81.2%. A model trained on the two baseline metabotypes present before intubation exhibited similar performance in the validation cohort. This study demonstrates the plausibility and promise of rapid metabolome-based prognostic discovery and validation in the opening wave of a pandemic. The approach used here could be used to inform therapeutic and resource allocation decisions early in a future epidemic.

IMPORTANCE

In a new respiratory virus pandemic, the ability to identify patients at greatest risk for severe disease is essential to direct scarce medical resources to those most likely to benefit from them. Tools to predict disease severity are best developed early in a pandemic, but laboratory-based resources to develop these may be limited by available technology and by infection precautions. Here, we show that an accessible metabolic profiling approach could identify a prognostic signature of severe disease in the initial wave of COVID-19, when patients presenting for care often exceeded the available doses of convalescent plasma and remdesivir. In a future pandemic, this approach, alongside efforts to identify clinical disease severity predictors, could improve patient outcomes and facilitate therapeutic trials by identifying individuals at high risk for severe disease.

INTRODUCTION

The arrival of SARS-CoV-2 over a short interval from late 2019 to early 2020 overwhelmed medical care providers worldwide. The rapid development of SARS-CoV-2 assays was a key element of the pandemic response, first for limiting transmission and soon after for identifying patients that could benefit from convalescent plasma and remdesivir therapy. Optimal allocation of these initially scarce agents required physicians to decide which patients were most likely to benefit. Estimating patient risk was further pressured by the need to administer antiviral agents early, often before respiratory failure was evident (16), as is typical of respiratory virus infections (7). As we endeavor to minimize morbidity and mortality in the next pandemic, rapidly ascertaining disease risk is an important area of emphasis.
Early in the COVID-19 pandemic, disease progression risk understandably relied upon readily accessible baseline clinical characteristics such as age, sex, or diabetes diagnosis (8). A recent retrospective chart review of 969 inpatients during the early pandemic indicates that further prognostic information was accessible from additional clinical data beyond baseline clinical characteristics (9). New laboratory biomarkers of risk also have potential to improve prognostic accuracy, but with a new disease, it is difficult to ascertain which of the thousands of possible analytes are most useful. One potential source of risk-associated laboratory biomarkers is the metabolome, the population of small molecules whose presence and abundance in patients reflect a broad array of physiological processes and exposures (10). Of note, clinical and research laboratories currently have the potential to expeditiously identify and deploy metabolome-derived prognostic signatures using the same liquid chromatography-mass spectrometry (LC-MS) profiling approaches currently applied to identify inborn errors of metabolism or drug and toxin exposures. These resources could be useful early in an epidemic when resource limitations imposed by geography and infectious agent precautions (specimen shipping, phlebotomist exposure, etc.) necessitate on-site discovery and validation.
In this study, we used LC-MS to identify metabolomic correlates of severe COVID-19 progression in patients presenting for evaluation at a U.S. academic medical center during local SARS-CoV-2 arrival, prior to vaccine availability. We obtained patient urine specimens upon emergency department admission and assessed their temporal association with severity-defining endpoints of respiratory failure or death. A metabolomic signature associated with severe COVID-19 progression in subjects enrolled early in the study was validated via blinded analysis in later enrollees. The severity-associated signature is composed of three metabotypes, two of which preceded onset of severe disease and one that preferentially appeared after intubation. This prototype assay exhibits a stronger prognostic potential for severe outcomes than a panel of clinical risk factors.

RESULTS

Clinical outcomes and SARS-CoV-2+ cohorts

Between 1 March 2020 and 30 September 2020, 163 patients presenting to the Barnes-Jewish Hospital emergency department with symptoms compatible with COVID-19 or at high risk of developing COVID-19 were enrolled (Fig. 1). Among the 150 SARS-CoV-2-positive patients in our cohorts, 45% (68) reached the pre-defined, severe COVID-19 composite endpoint of intubation or death by 90 days from admission, with all but 2 within 30 days. Of these, 82% (56) was intubated, with 50% (11) mortality post-intubation. Of the 40 COVID-19 patients who died during the study period, death was attributed to COVID-19 in 88%. Cause of death was not available for two COVID-19 patients. Patients with severe COVID-19 were significantly older (median 68.4 vs. 61.2 years, P < 0.001) with non-significantly increased diagnoses of diabetes or chronic kidney disease (36.8% vs. 25.6% and 7.4% vs. 2.4%, respectively; Table 1).
Fig 1
A flowchart displays the stepwise approach for metabolomic model development and validation, including subject enrollment, cohort selection, and the analysis process in a metabolomic study involving 163 subjects.
Fig 1 Study design and enrollment.
TABLE 1
TABLE 1 SARS-CoV-2 PCR-positive patient characteristics
 Non-severe COVID-19
N = 82
Severe COVID-19
N = 68
P valuea
Demographics
 Male51 (62.2%)42 (61.8%)>0.99
 Median age (interquartile range)61.2 (43.0–69.3)68.4 (59.9–76.0)<0.001b
 African American59 (71.1%)45 (66.2%)0.48
 Discovery cohort49 (59.8%)38 (55.9%)0.74
Patient comorbidities
 Malignancy or solid organ tumor4 (4.88%)2 (2.94%)0.69
 Diabetes mellitus21 (25.6%)25 (36.8%)0.16
 Chronic kidney disease2 (2.44%)5 (7.35%)0.25
 Asthma or chronic obstructive pulmonary disease15 (18.3%)8 (11.8%)0.36
 Congestive heart failure10 (12.2%)4 (5.88%)0.26
COVID-19 outcomes
 Respiratory failure during index hospitalization0 (0%)56 (82.4%)
 Death due to COVID-190 (0%)35 (51.5%)c
a
Fisher’s exact test, except where indicated.
b
Wilcoxon rank-sum test (boldface type indicates statistical significance).
c
Attribution of cause of death was not available for two deceased PCR-positive patients.
Discovery cohort enrollment ended after 4 months (1 March 2020 to 30 June 2020) with 96 patients. Subsequently, 67 patients were recruited (1 July 2020 to 30 September 2020) to the validation cohort. No significant differences in baseline clinical characteristics were observed between cohorts. When 90-day endpoints were known, we divided each cohort into severe and non-severe patients (39 and 57 for discovery, 30 and 37 for validation). Among severe COVID-19 patients, 57% (39/68) was intubated before urine collection while 50% of their urine specimens was collected between 2.9 days before and 3.5 days after reaching the severity-defining endpoint. Restricting to patients with positive SARS-CoV-2 PCR tests later reduced the discovery cohort to 87 patients (38 severe, 49 non-severe) and the validation cohort to 63 patients (30 severe, 33 non-severe).

Metabolomic correlates of progression to severe COVID-19

To identify metabolomic variation associated with progression to severe COVID-19, we obtained full-scan LC-MS profiles of the 96 discovery cohort samples (including PCR-negative samples) in positive- and negative-ion modes. Both unsupervised hierarchical clustering and principal component analysis (PCA) of low-sparsity metabolite logratios could partially discriminate severe and non-severe COVID-19 patient groups (Fig. S17 and S18). To better identify correlates of progression to severe COVID-19, we next applied multiple feature filtering and bounded feature selection techniques (see Materials and Methods). Models combining both logratios and clinical characteristics generally outperformed models trained with either data type alone (Fig. S15). Models including positive ion and/or negative ion-derived logratios performed comparably. Using positive-ion mode logratios and clinical characteristics with a maximum of 70 metabolites in our bounded feature selection yielded models with a mean area under the curve (AUC) of 0.801 from, on average, 20.6 metabolites. The Hamming distance between optimized feature subsets defined a single most representative set of 24 metabolites and 4 clinical variables (age, presence of stroke/dementia/seizure disorder, confusion, and fatigue) for further biomarker development. When sample collection for the discovery cohort began, patients with both confirmed and suspected SARS-CoV-2 infection were enrolled due to limitations in SARS-CoV-2 diagnostic performance and availability. Early steps of feature selection were performed on this combined discovery cohort, which contained nine PCR-negative patients (with one severe outcome). We retained these PCR-negative patients to prevent outcome information from biasing our already-performed bounded feature selection. Subsequent LC-MS/MS detection and classifier training (below) on the discovery cohort utilized only the 87 PCR-positive discovery cohort patients.

Defining a signature of progression to severe COVID-19

We next sought to identify a simplified metabolomic signature of progression to severe COVID-19. To quantify the 24 candidate metabolites with greater sensitivity, specificity, and precision than the full-scan LC-MS mode used for metabolomic discovery, we re-measured each using LC-MS/MS detection (Table S2). Of the 24 metabolites, 5 (21%) were detected in all specimens and 16 (67%) were detected in more than half. As detailed above, we used the DiCoVarML R package (12, 13) to train a model in terms of logratios between five metabolites, yielding the highest average training set AUCs among models trained with four to eight metabolites (Fig. S16).
Because ensemble models offered only modestly higher training AUCs, we chose to use more-interpretable ridge-regularized logistic regression (Fig. S16), yielding a model defined by 6 logratios between five metabolites (labeled by m/z and retention time as [email protected], [email protected], [email protected], [email protected], and [email protected]). Algebraic simplification from 6 logratios to the five underlying (log-transformed) metabolites yields regression coefficients for our final scoring function (Table 2). Model predictions are shown in Fig. 2A. The final model yielded an AUC of 89.4% (95% CI: 82.3–95.1) on the discovery cohort. The logratio derivation ensures the signature is inherently independent of absolute quantities and remains useful if sample dilution and/or instrument responses differ from those of the present study, provided the relative quantities of these five metabolites are accurately obtained.
TABLE 2
TABLE 2 Logistic regression coefficients in the five-feature model
Molecular featurePositionCoefficienta
[email protected]Numerator0.962
[email protected]Numerator0.793
[email protected]Numerator0.650
[email protected]Denominator−1.006
[email protected]Denominator−1.400
a
Coefficients do not sum to zero due to rounding.
Fig 2
Dot and box plots depict severe COVID-19 outcome probabilities in discovery and validation cohorts. ROC curves compare sensitivity and specificity, with 89.4% area under the curve for discovery and 81.2% for validation.
Fig 2 Logistic regression model on a subset of feature logratios confirms association between metabolite logratio signature and COVID-19 disease severity. (A) Dot plots of final-model-predicted probabilities that patients reached the severe endpoint, comparing probability distributions between the discovery and validation cohorts (two-sample Kolmogorov-Smirnov test). (B) Receiver operating characteristic (ROC) curves, AUCs, and bootstrapped AUC 95% confidence intervals for final logistic regression model predictions on the discovery (blue) and validation (black) cohorts.

Signature validation

To evaluate its prospective external validity, we applied our severe COVID-19 signature model to the validation cohort. Instrumental analyses and data interpretation were conducted by operators blinded to the outcomes associated with each specimen. Upon unblinding, the validation cohort AUC for the severe COVID-19 signature (81.2%; 95% CI: 68.4–91.1) was significantly better than the null hypothesis (50%) (Fig. 2B) and was not significantly lower than in the discovery cohort (P > 0.05, bootstrap test; Fig. 3) (14). We observed similar distributions (P > 0.05, Kolmogorov-Smirnov test) of outcome classification probabilities in discovery and validation cohorts (Fig. 2A). Univariate analysis of individual logratios also revealed similar trends between discovery and validation cohorts (Fig. 4B; Fig. S19). Using a standard 50% probability cutoff to assign regression classifications yielded similar accuracy on the discovery and validation cohorts (81.6% vs. 74.6%; Table S3). Together, these results indicate minimal model overfitting and strong beyond-sample generalizability of the severe COVID-19 metabolomic signature.
Fig 3
A forest plot compares ROC AUC values for DCV, metabotypes 1/2, metabotypes 1/2 plus clinical, DCV plus clinical, and clinical models across validation and discovery cohorts.
Fig 3 Performance of the final DiCoVar model (DCV) in the discovery and validation cohorts compared with models including clinical characteristics and with models excluding metabotype 3 (metabotypes 1/2). ROC AUC comparisons and 95% confidence intervals were computed by bootstrapping and corrected for multiple comparisons using Bonferroni’s method (20 comparisons within cohort, 5 comparisons between cohorts). Only significant comparisons are displayed (*P < 0.05, **P < 0.01). Vertical dashed line indicates 50% ROC AUC.
Fig 4
A network diagram shows associations between metabolites. A dot plot compares outcomes in discovery and validation cohorts. A heatmap shows hierarchical clustering of metabolite levels. A Venn diagram shows shared and unique metabotypes among patients.
Fig 4 (A) Network representation of logratios in the final five-feature logistic regression model. Edges point from denominator to numerator. Edge thickness and vertex area represent absolute logistic regression coefficients for logratios and individual features, respectively. Red and blue vertices are associated with severe (positive coefficients) and non-severe (negative coefficients) outcomes, respectively. (B) Peak areas of biomarker metabolites in discovery and validation cohort samples as measured by LC-MS/MS. Dashed line indicates mass spectrometer limit of detection at 1,000 counts. (C) Heatmap of prognostic logratios for discovery and validation cohort samples, with row and column dendrograms based on hierarchical clustering with Euclidean distance and complete (maximum) linkage. Color scales are normalized for each row. Red and blue column headers indicate patients with severe and non-severe outcomes, respectively. (D) Euler diagram illustrating the degree of overlap between patient metabotypes. Membership was defined by the presence of each numerator metabolite over the 1,000-count limit of detection.

Three metabotypes define the metabolomic signature

A network representation (Fig. 4A) of the logratios between the five validated features distinguishes their roles as numerators (3 features) and denominators (2 features) in the model. The two denominator features were present in all specimens and are negatively associated with severe COVID-19 (Wilcoxon test, P < 0.01; Fig. 4B ), although the relative magnitudes of these associations are small. The denominators thus effectively serve as “housekeeping” features, providing ubiquitous physiological references for metabolite normalization, analogous to creatinine. In contrast, each of the three numerator features exhibits bimodal distributions, consistent with three distinctive severity-associated metabotypes (Fig. 4B) (15). Metabotype 1 is defined by the presence of [email protected], metabotype 2 by [email protected], and metabotype 3 by [email protected]. Consistent with the manner of their selection, these metabotypes were not completely disjointed—subjects may simultaneously test positive for one or more metabotypes (Fig. 4C and D). Interestingly, metabotype 2 was associated with more advanced age (Wilcoxon test, P < 0.01; Fig. S20). No metabotype was associated with known clinical risk factors of diabetes or kidney disease (P > 0.05). These results are consistent with three independent biomarkers of COVID-19 severity with distinctive biochemical origins.

Temporal relationships between metabotypes and severe COVID-19

The potential for a metabotype to guide early antiviral interventions and patient disposition is influenced by its temporal relationship to severe disease, but this has not been typically considered in COVID-19 biomarker studies. Among the three metabotypes, the metabotype 3 signal was greater in specimens collected after intubation (Wilcoxon test, P < 0.01; Fig. 5). Metabotype 3 was neither uniformly absent in pre-intubation subjects nor uniformly present in post-intubation subjects, consistent with a physiological process that is closely related to acute respiratory failure. The presence of metabotypes 1 and 2, in contrast, was independent of intubation, consistent with baseline correlates of COVID-19 severity.
Fig 5
Dot plots display log-transformed values of urine metabolite ratios measured before and after clinical outcomes. Each panel represents a specific ratio, with data points for individual samples grouped as before outcome and after outcome.
Fig 5 Model logratios with numerators [email protected] and [email protected] are not associated with urine collection date relative to outcome date, but numerator [email protected] is significantly elevated in patients with urines collected after outcome (Wilcoxon rank-sum test; ***P < 0.001, not significant [NS.] P > 0.05).

Prognostic potential of metabotypes

COVID-19-associated metabotypes present early in infection, before the full inflammatory phase of illness, could potentially affect decisions about patient disposition and antiviral treatment. To evaluate their potential in this role, we fit predictor models including metabotypes 1 and 2 with or without baseline clinical predictor variables (diabetes, kidney disease, and age). The model using only metabotypes 1 and 2 was not significantly inferior to the model using all three metabotypes (Fig. 3, validation AUC 74.2% vs 81.2%; P > 0.05, bootstrap test with Bonferroni correction), consistent with significant predictive power in the absence of metabotype 3 (14). Adding clinical predictors non-significantly lowered the validation AUC versus metabotypes 1 and 2 alone. Notably, the validation cohort performance of clinical predictors alone was not significantly different from chance. These results suggest that metabotypes 1 and 2 are more consistently associated with severe COVID-19 than baseline clinical predictors.

Chemical identification of housekeeper metabolites used as denominators

Additional information about COVID-19 progression may be obtained by chemically identifying the metabolites in the severe COVID-19 signature. Multistage fragmentation (MSn) coupled with high-resolution accurate mass analysis of the five metabolites comprising our signature revealed they are oxyhydrocarbons, three of which possess a single nitrogen and no other heteroatoms. The two denominator metabolites in our model were identified with high confidence as an O-acylcarnitine ([email protected]) and δ-valerolactam ([email protected]) (Table 3). The fragmentation pattern of feature [email protected] contains multiple diagnostic fragments observed in other acylcarnitines (Fig. S2 to S5), with differences attributable to the presence of an unsaturated nine-carbon fatty acyl group. Feature [email protected] was definitively confirmed by comparison to authentic δ-valerolactam, an established urinary metabolite generated from cadaverine and subject to cytochrome P450-dependent metabolism (Fig. S6) (16).
TABLE 3
TABLE 3 Final prognostic model features and putative identification confidence
FeatureConfidencePutative identityESI + massRT (min)Neutral composition
[email protected]Confident class IDO-Nonanoylcarnitine302.232715.94C16H31NO4
[email protected]Confident IDLevetiracetam metabolite126.09065.04C7H11NO
[email protected]Approximate chemical classSimilar to 1-phenyl-2-hexanone177.1270410.56C12H16O
[email protected]Approximate chemical classSimilar to hydroxyenterodiol319.15220.1C18H22O5
[email protected]Confirmed with reference std.Delta-valerolactam100.07571.6C5H9NO

Metabotype 1 metabolite identification

The metabolite-defining metabotype 1, [email protected], was identified with high confidence as an electrospray ionization source-decay fragment of a previously-observed deaminated metabolite of the anticonvulsant drug levetiracetam (Keppra) (Fig. S7 to S10) (17). Its MS2 spectrum exactly matched a reference spectrum of levetiracetam in the mzCloud spectral library, while its molecular weight matches the deaminated metabolite. Concomitant detection of unmetabolized levetiracetam in specimens with this feature further increased confidence in this assignment (Fig. S10). This result is consistent with a positive association between levetiracetam use and progression to severe COVID-19.

Metabotype 2 metabolite identification

The metabotype 2 feature, [email protected], has predicted molecular formula C12H16O (Table 3) and shares multiple MS2 fragments with the fecal metabolite 1-phenyl-2-hexanone (Fig. S11A), including a tropylium ion characteristic of carbon-substituted benzene rings. Moreover, the 131.0850 m/z MS2 fragment ion has a neutral loss of C6H6 (consistent with benzene) in its MS3 spectrum (Fig. S11B). An H2O neutral loss in the MS2 spectrum also suggests the presence of an alcohol. Together, these data are consistent with [email protected] as a monounsaturated aliphatic alcohol-substituted benzene. The specific biochemical origin of this metabolite is unclear, though the presence of a benzene ring may implicate phenylalanine or tyrosine metabolism.

Metabotype 3 metabolite identification

The metabotype 3 feature, [email protected], with predicted molecular formula C18H22O5, yielded no direct match in online spectral databases. However, it shares distinctive fragments and neutral losses with enterodiol (Fig. S12 to S14), the eponymous member of an established group of human urinary metabolites and a product of lignan metabolism by the intestinal microbiota. The predicted formula (Table 3) matches hydroxyenterodiol, which may derive from P450-mediated hydroxylation of enterodiol in mammalian hosts (18, 19). These data are consistent with [email protected] as a member of the enterodiol family of bacterial-human co-metabolites derived from common dietary compounds. Elevation of this metabolite after respiratory failure in a subset of patients with severe COVID-19 may reflect increased systemic absorption associated with increased gut barrier permeability and/or increased human P450 metabolism of xenometabolites.

DISCUSSION

In this study, we describe an accessible, patient-based approach to rapid prognostic LC-MS/MS assay development for a new pandemic infection. If completed in time for use with COVID-19 patients in 2020, this approach might have improved triage decisions and optimized allocation of limited convalescent plasma and remdesivir. Entities charged with pandemic preparedness should consider expanding diagnostic efforts beyond pathogen identification alone to include prognostic assays compatible with existing laboratory equipment.
Relative to previously published metabolomic studies in COVID-19 patients, this study is distinguished by its clinically valid cohorts of patients presenting for medical evaluation, use of instrumentation accessible to clinical laboratories, feature selection yielding a simplified metabolite signature, blinded prospective validation, and careful attention to the relationship between metabolites and endpoint timing (2023). In the context of a new pandemic, accessibility of specimens prior to severe symptoms is challenging. With limited resources, we elected to implement this study in a hospital setting where tests were available, specimens could be safely collected, and outcomes could be reliably discerned. Because we reasoned that stable predictive markers would persist in patients after intubation, we elected to include subjects with post-intubation specimens in our cohorts. Inclusion of these specimens ensured timely recruitment of subjects and permitted temporal analyses (Fig. 5) to identify lagging, non-predictive, indicators such as metabotype 3. When the 20 post-intubation specimens (32% of subjects) were excluded from the validation cohort, the model exhibited a non-significant trend (receiver-operator characteristic area under the curve [ROC AUC] 68.2%, 95% CI: 49.4–84.6) toward prediction of severe outcomes. Given that metabotypes 1 and 2 are not temporally associated with intubation (Fig. 5), we regard this result as a probable Type II error. We advise future biomarker studies to carefully monitor the timing of specimen collection relative to key events in infection pathophysiology. Future studies may also consider longer term follow-up to identify markers for post-infectious sequelae. Any of the three severity-associated metabotypes identified here could conceivably modify the risk of post-COVID conditions, which are affected by the course of early infection (2427).
Several aspects of our approach are critical to its utility in a future pandemic. Primary among these is prompt establishment of an approved protocol for collecting specimens and longitudinal clinical characteristics. In the present study, this began in early March 2020, anticipating a surge of cases and a limited supply of antiviral therapies. The delay of the present study resulted in part from the recent development of the data analysis methods used here (12, 13, 28) and the need to curate the patient data, though other methods could have potentially been considered (11, 29). The present report was also delayed by efforts to determine the identities of metabolites used in the prototype assay, which we felt would be useful for interpreting the results. We calculate that our present approach would require 100–200 patients in a similar future pandemic, assuming that prognostic metabolites exist and exhibit log-normal distributions (Fig. S24). More advanced mass spectrometers with reduced data variability could reduce study size requirements. Specimen accrual and disease-specific patient data collection are likely to be rate-limiting steps.
Each of the three severe COVID-19 metabotypes suggests unique insights into physiologic processes or exposures that affect COVID-19 severity. The levetiracetam metabolite defining metabotype 1 was unexpected and raises important questions about its association with COVID-19 severity. Of the two subjects newly treated with levetiracetam prior to sample collection, only one had generalized seizure attributed to COVID-19. The majority (22/24, 92%) of subjects with detectable levetiracetam metabolite had been receiving it for seizure prophylaxis. Levetiracetam may be a marker for patient conditions that predispose to COVID-19 progression, a direct biological modifier of COVID-19 progression risk, or a combination thereof. Indeed, early levetiracetam efficacy trials conducted before the COVID-19 pandemic identified a positive association between levetiracetam administration and other respiratory infections (30). The levetiracetam metabolite does not exclusively drive the present model, which remains accurate even after holding out patients in whom it was detected (Fig. S21). Further study, such as a retrospective patient data analysis, would be necessary to determine the degree to which seizure disorders or related conditions, levetiracetam, or a class effect among anticonvulsants contribute to COVID-19 severity risk.
The nature of metabotype 2 remains unclear. While previously detected in human feces (31), 1-phenyl-2-hexanone is not clearly an intestinal or microbiome-derived product. Notably, metabotype 2 exhibits a distinctive association with age, a frequently applied proxy for COVID-19 risk. Its trend toward superiority over age as a severity predictor (Fig. S25) raises the possibility that it represents an age-associated physiological process that modifies COVID-19 risk. Given the widely noted role of age as a risk factor in the COVID-19 pandemic, the nature of this metabotype merits additional study.
Because metabotype 3 was preferentially detected in patients with established respiratory failure, we did not regard it as a strong prognostic candidate. This affects the metabotype’s potential to inform COVID-19 antiviral therapy, which is effective in most patients only when administered early in disease, as with other acute respiratory viruses (4, 6, 32, 33). Hydroxyenterodiol, the tentatively-identified feature defining metabotype 3, is derived from enterodiol, which is produced by the intestinal microbiota. It is plausible that intestinal inflammation in some forms of severe COVID-19 leads to increased enterodiol permeability and increased hydroxylation by host P450 enzymes. If metabotype 3 represents an inflammatory endotype in COVID-19 patients, it may instead help inform selection of immune modulating therapies (34, 35).
Limitations of this single-site study include its generalizability to other sites and its prospective applicability, both likely concerns for any evolving pandemic illness. At the time of this writing, population-wide vaccine and convalescent immunity to SARS-CoV-2 and antiviral supply is more robust than that in 2020, limiting the current clinical value of the prototype assay. This could change rapidly if a new SARS-CoV-2 variant emerges that evades current immunity or antiviral agents. Continued virus evolution and improved treatment strategies may also lessen the prospective validity of an assay based on results from a single site early in the pandemic. An intrinsic limitation of metabolomic data analysis is that choices made during machine learning may influence the set of biomarkers discovered, especially when multiple near-optimal sets exist. Preliminary tests (Fig. S22 and S23) suggest that the biomarker signature presented here is relatively robust to these choices. More sophisticated but less widely available instrumentation may have identified different or more complex signatures with superior sensitivity or specificity but more limited applicability. In a future pandemic, some of these limitations may be remedied by coordination between sites and access to improved instrumentation.

MATERIALS AND METHODS

Patient population

Patients presenting to the Barnes-Jewish Hospital emergency department with symptoms compatible with COVID-19 or at high risk of developing COVID-19 were enrolled upon submission of a nasopharyngeal swab for SARS-CoV-2 RNA testing using a Food and Drug Administration-approved clinical PCR test (Fig. S1). The primary endpoint for progression of COVID-19 to severe disease was respiratory failure requiring mechanical ventilation and/or death within 90 days of study enrollment. Subjects enrolled from 1 March 2020 to 30 June 2020 comprised the initial discovery data set (n = 96, SARS-CoV-2 positive n = 87). Additional subjects enrolled between 1 July 2020 and 30 September 2020 comprised the validation data set (n = 67, SARS-CoV-2 positive n = 63) (Table 1). The workflow for discovery and validation cohorts is outlined in Fig. S1. Importantly, no information about the validation cohort was used in identifying and formulating our candidate metabolomic signature. Clinical data were extracted from electronic health records by the Washington University Institute of Informatics and provided in a de-identified manner.

Urine collection and processing

Patient urine samples were collected in a sterile cup by the hospital staff as convenient and transported twice a day to the laboratory. Samples were stored at 4°C until ready for processing on the same day. All samples were processed and stored in less than 24 hours. Urine was filtered through a 40-µm mesh followed by 2,000 × g centrifugation for 12 minutes. Supernatant was immediately aliquoted into cryovials and stored at −80°C until ready for experimental analysis.

Liquid chromatography-mass spectrometry

Urinary metabolites were measured using an AB Sciex 4000 QTrap triple-quadrupole mass spectrometer interfaced with a Shimadzu HPLC as previously described (36, 37), (see supplemental material for details). Reversed-phase chromatography was conducted with an Ascentis-Express fused core phenyl-hexyl column (Millipore Sigma; 100 mm × 2 mm × 2.7 µm). Metabolite profiling was conducted in full-scan mode across a mass/charge range of 50–1,200 m/z. Multiplexed profiling of selected metabolites was conducted in MS/MS mode. Structural characterization of validated metabolites was performed using an Orbitrap ID-X high-resolution accurate mass spectrometer coupled to a Vanquish ultra-high pressure liquid chromatograph (UHPLC; Thermo Scientific), as described in the supplemental materials (Fig. S2 to S14).

Metabolomic feature filtering and expression as logratios

To focus our search on broadly applicable features, we restricted analyses to the 103 negative-ion and 163 positive-ion features present in >50% of samples in at least one clinical outcome group. To control for physiologic differences in urinary dilution, we analyzed urinary metabolomic profiles as log-transformed metabolite peak area ratios (hereafter designated “logratios”) and selected from all possible ratios in the data during subsequent feature selection steps (11, 29). Using metabolite ratios allowed us to identify urinary reference metabolites without making a priori assumptions, for example, we did not assume that any single metabolite, such as creatinine, would provide an optimal physiologic reference.

Bounded feature selection and machine learning modeling

To select a reduced set of LC-MS metabolite features for targeted LC-MS/MS analysis, we used a feature selection approach based in genetic algorithms (for additional details, see supplemental materials). To estimate the ability of metabolomic and/or clinical data to correctly classify severe outcomes and to minimize overfitting, we performed this process with fivefold cross-validation train/test splits of the discovery cohort, repeated 20 times with different random seeds, for each combination of metabolomic data and clinical metadata. After a subset of metabolites was identified using the genetic algorithm approach, we used differential compositional variation scores to select subsets of logratios for model training through multistage feature selection using network-based, recursive elimination combined with penalized regression techniques (28). This broad approach was used to improve model performance and improve interpretability by eliminating uninformative logratios. Clinical metadata features were optionally selected using the Boruta algorithm on individual training partitions (38). Features selected in 8 of 10 repeats of the Boruta algorithm were retained. The resulting clinical metadata subset was then appended to the selected metabolite subset. Finally, the XGBoost algorithm was used to train tree booster models using the selected features (39).
Across all test splits, average model AUC was superior for models combining metabolite and clinical data. Models using positive-mode data, negative-mode data, or both modes yielded similar average model AUCs, with larger models generally exhibiting higher AUCs (Fig. S15). Because positive mode ionization is more common and sensitive, we proceeded with a model combining positive mode features and clinical metadata. To select a smaller set of representative metabolites and clinical attributes, we repeated the feature selection process 50 times with LC-MS features and clinical attributes, with a maximum model size of 70 variables. We aggregated the importance scores for each selected variable across all models and, representing each subset as a binary vector, selected the subset with minimum total pairwise Hamming distances as the most representative (Fig. S22). The 24 metabolites in this most representative subset were selected for targeted metabolomic analysis.

Metabolite signature selection and final prognostic model

To simplify the 24-metabolite signature while maintaining classification performance, we used the targeted differential compositional variation machine learning framework implemented in the DiCoVarML R package (13). DiCoVarML is a multilevel nested framework that respects the variable dilution of urinary specimens by leveraging network methods that identify and robustly estimate classification performance of minimal logratio signatures (13). Pairwise logratios between the 24 metabolites were calculated with zeroes imputed using multiplicative replacement (28, 40). We selected ridge regression over ensemble models for easier interpretability and because we did not see substantial decrease in accuracy (Fig. S16). As in the previous stages of feature selection and model building, only discovery cohort data were used as input. Out-of-sample classification performance was estimated using five repeats of twofold cross-validation for each target subset size (four to eight metabolites). Multiplicative zero imputation was performed separately for each cross-validation step to avoid leaking information between folds. The DiCoVarML nested classification performance estimate was then used to select the final subset of five metabolites (Fig. S16) and train the final ridge regression model with the corresponding pairwise logratios. For ease of interpretation, the model in terms of 6 logratios was algebraically simplified to 5 log-scaled metabolite signal values to specify our final prognostic function (coefficients in Table 2). This final prognostic function was applied to LC-MS/MS data to predict outcome severity in the validation cohort.

Statistics

All statistical analysis and machine learning model building and assessment were done in R version 4.1.0. The R mixOmics package was used for PCA (41). We used the R glmnet package to predict outcome probabilities and labels using the final model trained on the discovery data (42). Discovery and validation set AUCs and 95% confidence intervals were computed using the R pROC package (14). ROC AUCs were compared using bootstrapping (R pROC package, roc.test function). Comparison P values were adjusted for multiple comparisons by the Bonferroni method (R stats package, p.adjust function). The prognostic function scores, using the discovery-trained model, were computed for both validation and discovery data using the R predict function with response output. The Kolmogorov-Smirnov test (R stats package, function ks.test) was used to compare distributions of predicted probabilities of severe outcome between discovery and validation cohorts, stratified by outcome. The two-sample Wilcoxon signed-rank test (R stats package, function wilcox.test) was used to describe differences in means of logratio features between outcome groups.

ACKNOWLEDGMENTS

This study utilized samples from the Washington University School of Medicine’s COVID-19 biorepository, which is supported by the following: the Barnes-Jewish Hospital Foundation, the Siteman Cancer Center grant P30 CA091842 from the National Cancer Institute of the National Institutes of Health, and the Washington University Institute of Clinical and Translational Sciences grant UL1TR002345 from the National Center for Advancing Translational Sciences of the National Institutes of Health. This research was also supported by The Longer Life Foundation (J.P.H.), National Institutes of Health RO1DK111930 (J.P.H.), an HHMI Gilliam Award (GT11504, A.L.H. and P.J.M.), and a James S. McDonnell Foundation Complex Systems Scholar Award (#220020315, P.J.M.). The content is solely the responsibility of the authors and does not necessarily represent the view of the NIH or any other agency funding this research.
We are grateful to Adriana Rauseo Acevedo and Rachel Presti for their efforts in data management and patient recruitment.
J.P.H., L.R.M., and J.I.R. developed the original concept and designed the overall study approach. J.A.O. and C.W.G. conducted enrollment, specimen collection, and clinical data collection. L.R.M. and J.I.R. conducted specimen processing and mass spectrometric analyses, prepared the data, and interpreted spectra. J.I.R., A.L.H., L.R.M., P.J.M., and J.P.H. analyzed the data and wrote the manuscript.

SUPPLEMENTAL MATERIAL

Supplemental Material - mbio.03343-23-s0001.docx
Supplemental methods, Fig. S1 to S25, and Tables S1 to S3.
ASM does not own the copyrights to Supplemental Material that may be linked to, or accessed through, an article. The authors have granted ASM a non-exclusive, world-wide license to publish the Supplemental Material files. Please contact the corresponding author directly for reuse.

REFERENCES

1.
Salazar E, Christensen PA, Graviss EA, Nguyen DT, Castillo B, Chen J, Lopez BV, Eagar TN, Yi X, Zhao P, Rogers J, Shehabeldin A, Joseph D, Masud F, Leveque C, Olsen RJ, Bernard DW, Gollihar J, Musser JM. 2021. Significantly decreased mortality in a large cohort of coronavirus disease 2019 (COVID-19) patients transfused early with convalescent plasma containing high-titer anti-severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) spike protein IgG. Am J Pathol 191:90–107.
2.
Libster R, Pérez Marc G, Wappner D, Coviello S, Bianchi A, Braem V, Esteban I, Caballero MT, Wood C, Berrueta M, et al. 2021. Early high-titer plasma therapy to prevent severe COVID-19 in older adults. N Engl J Med 384:610–618.
3.
Joyner MJ, Carter RE, Senefeld JW, Klassen SA, Mills JR, Johnson PW, Theel ES, Wiggins CC, Bruno KA, Klompas AM, et al. 2021. Convalescent plasma antibody levels and the risk of death from COVID-19. N Engl J Med 384:1015–1027.
4.
Sullivan DJ, Gebo KA, Shoham S, Bloch EM, Lau B, Shenoy AG, Mosnaim GS, Gniadek TJ, Fukuta Y, Patel B, et al. 2022. Early outpatient treatment for COVID-19 with convalescent plasma. N Engl J Med 386:1700–1711.
5.
Brown SM, Katz MJ, Ginde AA, Juneja K, Ramchandani M, Schiffer JT, Vaca C, Gottlieb RL, Tian Y, Elboudwarej E, Hill JA, Gilson R, Rodriguez L, Hedskog C, Chen S, Montezuma-Rusca JM, Osinusi A, Paredes R. 2023. Consistent effects of early remdesivir on symptoms and disease progression across at-risk outpatient subgroups: treatment effect heterogeneity in PINETREE Study. Infect Dis Ther 12:1189–1203.
6.
Gupta A, Gonzalez-Rojas Y, Juarez E, Crespo Casal M, Moya J, Falci DR, Sarkis E, Solis J, Zheng H, Scott N, Cathcart AL, Hebner CM, Sager J, Mogalian E, Tipple C, Peppercorn A, Alexander E, Pang PS, Free A, Brinson C, Aldinger M, Shapiro AE, Investigators C-I. 2021. Early treatment for COVID-19 with SARS-CoV-2 neutralizing antibody sotrovimab. N Engl J Med 385:1941–1950.
7.
Koonin LM, Patel A. 2018. Timely antiviral administration during an influenza pandemic: key components. Am J Public Health 108:S215–S220.
8.
Park H, Tarpey T, Liu M, Goldfeld K, Wu Y, Wu D, Li Y, Zhang J, Ganguly D, Ray Y, et al. 2022. Development and validation of a treatment benefit index to identify hospitalized patients with COVID-19 who may benefit from convalescent plasma. JAMA Netw Open 5:e2147375.
9.
Natanov D, Avihai B, McDonnell E, Lee E, Cook B, Altomare N, Ko T, Chaia A, Munoz C, Ouellette S, Nyalakonda S, Cederbaum V, Parikh PD, Blaser MJ. 2023. Predicting COVID-19 prognosis in hospitalized patients based on early status. MBio 14:e0150823.
10.
Dettmer K, Aronov PA, Hammock BD. 2007. Mass spectrometry-based metabolomics. Mass Spectrom Rev 26:51–78.
11.
Greenacre M, Grunsky E, Bacon-Shone J, Erb I, Quinn T. 2022. Aitchison’s compositional data analysis 40 years on: a reappraisal. Statist Sci 38.
12.
Hinton AL, Mucha PJ. 2021. Differential compositional variation feature selection: a machine learning framework with log ratios for compositional metagenomic data. Bioinformatics.
13.
Hinton AL, Mucha PJ. 2021. DiCoVarML package. Available from: https://github.com/andrew84830813/DiCoVarML. Retrieved 7 Jul 2021.
14.
Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M. 2011. pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinformatics 12:77.
15.
Beger RD, Dunn W, Schmidt MA, Gross SS, Kirwan JA, Cascante M, Brennan L, Wishart DS, Oresic M, Hankemeier T, Broadhurst DI, Lane AN, Suhre K, Kastenmüller G, Sumner SJ, Thiele I, Fiehn O, Kaddurah-Daouk R, for “Precision Medicine and Pharmacometabolomics Task Group”-Metabolomics Society Initiative. 2016. Metabolomics enables precision medicine: “a white paper, community perspective.” Metabolomics (Los Angel) 12:149.
16.
Cheng J, Chen C, Kristopher KW, Manna SK, Scerba M, Friedman FK, Luecke H, Idle JR, Gonzalez FJ. 2013. Identification of 2-piperidone as a biomarker of CYP2E1 activity through metabolomic phenotyping. Toxicol Sci 135:37–47.
17.
Strolin Benedetti M, Whomsley R, Nicolas JM, Young C, Baltes E. 2003. Pharmacokinetics and metabolism of 14C-levetiracetam, a new antiepileptic agent, in healthy volunteers. Eur J Clin Pharmacol 59:621–630.
18.
Jacobs E, Kulling SE, Metzler M. 1999. Novel metabolites of the mammalian lignans enterolactone and enterodiol in human urine. J Steroid Biochem Mol Biol 68:211–218.
19.
Jacobs E, Metzler M. 1999. Oxidative metabolism of the mammalian lignans enterolactone and enterodiol by rat, pig, and human liver microsomes. J Agric Food Chem 47:1071–1077.
20.
Roberts I, Wright Muelas M, Taylor JM, Davison AS, Xu Y, Grixti JM, Gotts N, Sorokin A, Goodacre R, Kell DB. 2021. Untargeted metabolomics of COVID-19 patient serum reveals potential prognostic markers of both severity and outcome. Metabolomics (Los Angel) 18:6.
21.
Jing Y, Wang J, Zhang H, Yang K, Li J, Zhao T, Liu J, Wu J, Chen Y. 2022. Alterations of urinary microbial metabolites and immune indexes linked with COVID-19 infection and prognosis. Front Immunol 13:841739.
22.
Li Y, Hou G, Zhou H, Wang Y, Tun HM, Zhu A, Zhao J, Xiao F, Lin S, Liu D, et al. 2021. Multi-platform omics analysis reveals molecular signature for COVID-19 pathogenesis, prognosis and drug target discovery. Sig Transduct Target Ther 6:155.
23.
Battaglini D, Lopes-Pacheco M, Castro-Faria-Neto HC, Pelosi P, Rocco PRM. 2022. Laboratory biomarkers for diagnosis and prognosis in COVID-19. Front Immunol 13:857573.
24.
Blomberg B, Mohn KG-I, Brokstad KA, Zhou F, Linchausen DW, Hansen B-A, Lartey S, Onyango TB, Kuwelker K, Sævik M, Bartsch H, Tøndel C, Kittang BR, Bergen COVID-19 Research Group, Cox RJ, Langeland N. 2021. Long COVID in a prospective cohort of home-isolated patients. Nat Med 27:1607–1613.
25.
Su Y, Yuan D, Chen DG, Ng RH, Wang K, Choi J, Li S, Hong S, Zhang R, Xie J, et al. 2022. Multiple early factors anticipate post-acute COVID-19 sequelae. Cell 185:881–895.
26.
Gebo KA, Heath SL, Fukuta Y, Zhu X, Baksh S, Abraham AG, Habtehyimer F, Shade D, Ruff J, Ram M, et al. 2023. Early antibody treatment, inflammation, and risk of post-COVID conditions. MBio 14:e0061823.
27.
Henderson JP. 2023. Igniting the slow burn of post-COVID conditions. MBio 14:e0148923.
28.
Hinton AL, Mucha PJ. 2022. A simultaneous feature selection and compositional association test for detecting sparse associations in high-dimensional metagenomic data. Front Microbiol 13:837396.
29.
Aitchison J. 1982. The statistical analysis of compositional data. J R Stat Soc Ser B 44:139–160.
30.
Mbizvo GK, Dixon P, Hutton JL, Marson AG. 2014. The adverse effects profile of levetiracetam in epilepsy: a more detailed look. Int J Neurosci 124:627–634.
31.
De Angelis M, Piccolo M, Vannini L, Siragusa S, De Giacomo A, Serrazzanetti DI, Cristofori F, Guerzoni ME, Gobbetti M, Francavilla R. 2013. Fecal microbiota and metabolome of children with autism and pervasive developmental disorder not otherwise specified. PLoS ONE 8:e76993.
32.
Hammond J, Leister-Tebbe H, Gardner A, Abreu P, Bao W, Wisemandle W, Baniecki M, Hendrick VM, Damle B, Simón-Campos A, Pypstra R, Rusnak JM, EPIC-HR Investigators. 2022. Oral nirmatrelvir for high-risk, nonhospitalized adults with COVID-19. N Engl J Med 386:1397–1408.
33.
Torres A, Motos A, Cillóniz C, Ceccato A, Fernández-Barat L, Gabarrús A, Bermejo-Martin J, Ferrer R, Riera J, Pérez-Arnal R, et al. 2022. Major candidate variables to guide personalised treatment with steroids in critically ill patients with COVID-19: CIBERESUCICOVID study. Intensive Care Med 48:850–864.
34.
Annane D, Meduri GU. 2022. Precision medicine for corticotherapy in COVID-19. Intensive Care Med 48:926–929.
35.
Sinha P, Furfaro D, Cummings MJ, Abrams D, Delucchi K, Maddali MV, He J, Thompson A, Murn M, Fountain J, Rosen A, Robbins-Juarez SY, Adan MA, Satish T, Madhavan M, Gupta A, Lyashchenko AK, Agerstrand C, Yip NH, Burkart KM, Beitler JR, Baldwin MR, Calfee CS, Brodie D, O’Donnell MR. 2021. Latent class analysis reveals COVID-19-related acute respiratory distress syndrome subgroups with differential responses to corticosteroids. Am J Respir Crit Care Med 204:1274–1285.
36.
Parker KS, Crowley JR, Stephens-Shields AJ, van Bokhoven A, Lucia MS, Lai HH, Andriole GL, Hooton TM, Mullins C, Henderson JP. 2016. Urinary metabolomics identifies a molecular correlate of interstitial cystitis/bladder pain syndrome in a multidisciplinary approach to the study of chronic pelvic pain (MAPP) research network cohort. EBioMedicine 7:167–174.
37.
Zou Z, Robinson JI, Steinberg LK, Henderson JP. 2023. Uropathogenic Escherichia coli wield enterobactin-derived catabolites as siderophores. J Biol Chem.
38.
Kursa MB, Rudnicki WR. 2010. Feature selection with the boruta package. J Stat Softw 36:1–13.
39.
Chen T, Guestrin C. 2016. XGBoost: a scalable tree boosting system. Abstr Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, San Francisco, California, USA
40.
Martín-Fernández J-A, Hron K, Templ M, Filzmoser P, Palarea-Albaladejo J. 2015. Bayesian-multiplicative treatment of count zeros in compositional data sets. Stat Modelling 15:134–158.
41.
Rohart F, Gautier B, Singh A, Lê Cao K-A. 2017. mixOmics: an R package for ’omics feature selection and multiple data integration. PLoS Comput Biol 13:e1005752.
42.
Friedman JH, Hastie T, Tibshirani R. 2010. Regularization paths for generalized linear models via coordinate descent. J Stat Soft 33:22.

Information & Contributors

Information

Published In

cover image mBio
mBio
Online First
eLocator: e03343-23
Editor: Martin J. Blaser, Rutgers The State University of New Jersey, Piscataway, New Jersey, USA
PubMed: 39576111

History

Received: 2 January 2024
Accepted: 29 October 2024
Published online: 22 November 2024

Keywords

  1. COVID-19
  2. metabolomics
  3. prognostic indicators

Data Availability

De-identified clinical metadata, processed metabolomic data, and associated data analysis scripts may be obtained from the corresponding author upon request.

Contributors

Authors

Division of Infectious Diseases, Department of Internal Medicine, Washington University School of Medicine, St. Louis, Missouri, USA
Author Contributions: Data curation, Formal analysis, Investigation, Methodology, Validation, and Writing – original draft.
Laura R. Marks
Division of Infectious Diseases, Department of Internal Medicine, Washington University School of Medicine, St. Louis, Missouri, USA
Author Contributions: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, and Validation.
Andrew L. Hinton
Curriculum in Bioinformatics and Computational Biology, University of North Carolina, Chapel Hill, North Carolina, USA
Author Contributions: Formal analysis, Investigation, and Methodology.
Jane A. O'Halloran
Division of Infectious Diseases, Department of Internal Medicine, Washington University School of Medicine, St. Louis, Missouri, USA
Author Contributions: Conceptualization, Data curation, Formal analysis, Investigation, and Methodology.
Charles W. Goss
Division of Biostatistics, Washington University School of Medicine, St. Louis, Missouri, USA
Peter J. Mucha
Curriculum in Bioinformatics and Computational Biology, University of North Carolina, Chapel Hill, North Carolina, USA
Department of Mathematics, Dartmouth College, Hanover, New Hampshire, USA
Author Contributions: Formal analysis, Investigation, Methodology, Project administration, and Writing – original draft.
Division of Infectious Diseases, Department of Internal Medicine, Washington University School of Medicine, St. Louis, Missouri, USA
Author Contributions: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, and Writing – original draft.

Editor

Martin J. Blaser
Editor
Rutgers The State University of New Jersey, Piscataway, New Jersey, USA

Notes

The authors declare no conflict of interest.

Ethics Approval

Samples and clinical data were collected from consented hospitalized patients at the Washington University Medical Center under an institutional review board-approved protocol to study immunological, genetic, and clinical predictors of SARS CoV-2 infection (IRB ID # 202003085) through assistance from the Washington University Institute of Clinical and Translational Sciences. All patients provided written, informed consent prior to participating in this study.

Metrics & Citations

Metrics

Note:

  • For recently published articles, the TOTAL download count will appear as zero until a new month starts.
  • There is a 3- to 4-day delay in article usage, so article usage will not appear immediately after publication.
  • Citation counts come from the Crossref Cited by service.

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

View Options

Figures and Media

Figures

Media

Tables

Share

Share

Share the article link

Share with email

Email a colleague

Share on social media

American Society for Microbiology ("ASM") is committed to maintaining your confidence and trust with respect to the information we collect from you on websites owned and operated by ASM ("ASM Web Sites") and other sources. This Privacy Policy sets forth the information we collect about you, how we use this information and the choices you have about how we use such information.
FIND OUT MORE about the privacy policy