Rapid inference of antibiotic susceptibility phenotype of uropathogens using metagenomic sequencing with neighbor typing

We first sought to develop regional reference databases of clinical isolates from ICUs in two large cities in Ontario for two Gram-negative pathogens of concern. We collected 131 (Klebsiella spp. including Klebsiella pneumoniae, Klebsiella variicola, Klebsiella oxytoca, and Klebsiella aerogenes) and 148 (E. coli) isolates for these databases (Table 1). These isolates were collected from urine samples that were culture-positive for one of the pathogens of interest whether they had resistance to any specific agent; these isolates, therefore, represent a sampling of the ICU population at the time of collection from August 2020 to January 2021. There were 106 multilocus sequence types (MLST) represented in the Klebsiella spp. database (58 K. pneumoniae, 25 K. oxytoca, 20 K. aerogenes, and 3 K. variicola), with 51 represented in the E. coli data set (Table 1). Commonly encountered E. coli MLSTs included those consistent with E. coli lineages known to cause a large proportion of human extraintestinal infections (21, 22). As expected, genetic trees for each reference set show clustering of related isolates by MLST (Fig. 1 and 2).

We sequenced metagenomic DNA from remnant primary urine specimens from ICUs in Toronto and Ottawa with ONT MinION. Following initial processing with Kraken2, we ran the RASE algorithm with the organism-specific regional reference database for E. coli and a genus-specific regional reference database for Klebsiella spp. based on the dominant uropathogen. Stability was achieved with a median of 203 (Q1-Q3: 161–345) reads for E. coli and 260 (Q1-Q3: 169–559) reads for Klebsiella spp. Time to stability occurred at a median of 9 (Q1-Q3: 5–28) and 19 (Q1-Q3: 7–114) min of sequencing time for E. coli and Klebsiella spp., respectively (Table S1). The median retained reads belonging to the putative pathogen of interest were 3281 (Q1-Q3: 2740, 3573) for E. coli and 3340 (Q1-Q3: 2186–3359) for Klebsiella spp., corresponding to approximately 82% and 83.5% reads, respectively (Table S1). We had 88 initial metagenomic samples (69 E. coli and 19 Klebsiella spp.), of which 8 (9.1%) were removed, as they had dominant pathogens (by read count) that were discordant with the selected clinical uropathogen (by culture), and these were removed from the subsequent analysis (Table S2). Three samples (two with cultures yielding E. coli and one with cultures yielding Klebsiella spp.) had a non-uropathogen as dominant, but the expected uropathogen was the second most common contributor, and these reads were retained. Repeated taxonomic classification, of only reads available up to the time of stability, confirmed consistency with dominant pathogen classification and ensured reproducible classification could be made prospectively. All subsequent analyses were performed on all retained reads classified as the dominant pathogen only, unless otherwise specified.

We next assessed the MLST of the closest match predicted by RASE and compared it with the presumed true MLST (determined from the short-read sequence data from the isolates derived from the matching primary urine samples) (Table 2; Table S3). The samples were classified using the MLST package (23, 24). The correct prediction of sequence type by RASE was improved when stratifying by lineage score, which marks greater confidence in the best match lineage predictions (Table 2). A lineage score (LS) is provided for each call, with higher scores indicating increased confidence in lineage assignments, and any stratification for LS was done using LS >0.5. Concordance, the match between true and predicted MLST, for E. coli samples when the lineage score was greater than or equal to 0.5% was 76.2% compared with 40.9% with LS below 0.5. Concordance for Klebsiella spp. was 55.6% when the lineage score was greater than or equal to 0.5 compared with 14.2% with LS below 0.5. Further investigation found that low LS occurred more frequently when the corresponding MLST is absent in the database and thus may mediate discordant results (Table S3), with the exception of a few sequence types, such as ST1193 of E. coli, where the MLST is well-represented in the database (n = 14), but five of the six samples with MLST 1193 had LS <0.5.

To assess the ability to predict AMR, we computed the pre-test probability of susceptibility (or prevalent susceptibility) and the probability of true susceptibility for isolates predicted to be resistant or susceptible when using RASE (Fig. 3). Values higher than the pre-test probability and/or the thresholds of 80%–90% indicate that RASE predictions for susceptibility are more likely to be truly susceptible. Overall likelihood ratios for RASE-prediction for all antibiotic susceptibilities in each of E. coli and Klebsiella spp. are shown in Table 3, whereas the details of sensitivity, specificity, and negative and positive likelihood ratios for each antibiotic-pathogen pair are shown in Table S4. The pre-test (or prevalent) probabilities of susceptibility vary widely for E. coli (Fig. 3A), depending on the antibiotic, and are generally less susceptible, with more variation than for Klebsiella spp. (Fig. 3B). The probability of susceptibilities given either RASE-predicted non-susceptible or RASE-predicted susceptible are generally left and right of the pre-test probability, respectively, as expected. This is summarized in the pooled (overall) data, via sensitivity and specificity values as well as likelihood ratios (which are independent of baseline prevalence of resistance) of susceptibility. We found (without stratification) a sensitivity of 0.69 (95% CI: 0.64, 0.74) and a specificity of 0.65 (95% CI: 0.56, 0.73) as well as a positive likelihood ratio (LR+) of 1.96 (95% CI: 1.60, 2.40) and a negative likelihood ratio (LR-) of 0.48 (95% CI: 0.39, 0.58) (Table 3) for predicting susceptibility. We also assessed the change in probability of susceptibility, given a RASE prediction of susceptible or non-susceptible (Table 3). For the pooled data, the pre-test probability was 0.74, and the absolute change in predicted susceptibility for RASE-predicted non-susceptible isolates was a reduction of 16.6%, whereas the absolute change in susceptibility for RASE-predicted susceptible was an increase of 10.9%. RASE-predicted susceptibilities values were also evaluated to see if they were moved over relevant clinical treatment thresholds representing acceptable likelihood of susceptibility for low-severity (80%) and high-severity (90%) disease (25); in this case, the value for trimethoprim-sulfamethoxazole for Klebsiella spp. was moved above the 90% threshold. Notably, numerous agents (gentamicin, ciprofloxacin, piperacillin/tazobactam, ceftriaxone, and cefazolin) for both organism types had their probabilities of susceptibilities moved below typical treatment thresholds when there was a RASE-predicted non-susceptible prediction, which shows the potential value for identifying infections at high likelihood of resistance that may necessitate different antibiotic therapies. We also assessed if there were improvements to prediction using RASE with either local databases (Ottawa- and Toronto-specific), or a European (EuSCAPE) database, to determine if assessing region-specific samples to region-specific databases improved analysis or if broader, although geographically different, databases improved prediction. In the case of the local databases, these did not clearly perform any better than using the regional databases for both E. coli and Klebsiella spp. (Fig. S2; see Supplementary Results for expanded analysis). Using the European database produced similar results to the regional databases (Fig. S3) and yielded MLST concordance for some samples that had an MLST not represented in the regional databases (Table S8).

We assessed whether RASE-produced lineage and susceptibility scores could be used to improve prediction. The lineage score (LS) is provided for each call, with higher scores indicating increased confidence in lineage assignments. Susceptibility scores (SS) are provided for each antibiotic: scores range from 0 to 1, with lower values representing higher confidence for non-susceptible calls, and higher values representing higher confidence for susceptible calls; values between 0.4 and 0.6 represent calls where RASE is less confident that the phenotype prediction is in fact resistant or susceptible.

Stratifying by SS (SS <0.4 and SS >0.6), LS (LS >0.5), or both resulted in the loss of data for increasing percentages of antibiotic susceptibility calls. Although stratification by SS seemed to offer no obvious advantage, there was a suggestion of possible benefit of stratification by lineage score (Table 3). Sensitivities were higher for all levels of stratification, with values increasing the least when stratifying by SS and increasing the most with combined stratification by LS and SS. However, specificities were all the same or lower following stratification (Table 3). Thus, stratification by SS or LS has the potential to improve sensitivity, but not specificity.

We also assessed the performance when accounting for concordance of predicted and actual MLST. We hypothesized that the potential benefit of LS stratification might be mediated through the concordance of MLST. Although there was a trend suggesting increased accuracy of prediction for MLST concordant E. coli, this was not observed for Klebsiella spp. For E. coli, concordance improved LR + and LR- compared with discordant calls, but this was not observed with Klebsiella spp. (Table 3). Note that the pooled results are largely informed by E. coli, but this reflects the common relative distribution of the two pathogens in UTI. Similarly, concordance improved both the sensitivity and specificity for E. coli but had variable impacts for Klebsiella spp. (Table 3).

Although conclusions based on changes in test characteristics by stratification are limited by the size of the confidence intervals (Table 3), we can compare the impact on point estimates (Fig. 4). For E. coli, using the LR point estimates and comparing with a non-stratified approach, the LS stratification has the potential to support reconsidering of one additional (and commonly used) antibiotic across the 80% susceptibility threshold (ciprofloxacin). For Klebsiella spp. RASE-predicted susceptible probabilities were marginally improved with stratification, particularly for ceftriaxone (Fig. 4B). However, this did not reflect a major change in LR+ (Table 3). For E. coli and Klebsiella spp., we also separated the antibiotics by whether they were typically administered orally or intravenously and calculated the relevant test characteristics (Fig. S1). For E. coli, the RASE-predicted susceptible probability moved to 80% or greater for antibiotics typically administered intravenously (improving potential for empiric use) or orally. In Klebsiella spp., the RASE-predicted susceptible probability for orally administered antibiotics was also moved over the 80% threshold, whereas those typically administered intravenously were moved over the 90% threshold.

We performed a multicenter prospective study evaluating the role of rapid metagenomic sequencing combined with k-mer-based neighbor typing approaches for the purposes of rapidly predicting antibiotic resistance from clinical urine samples (Fig. 5). First, we prospectively generated RASE k-mer organism type reference databases for two Gram-negative pathogens isolated from critical care patients (E. coli and Klebsiella spp.). Database isolates were collected from samples submitted to clinical microbiology laboratories from critical care units at each of four participating tertiary care hospitals (two in Toronto and two in Ottawa) in Ontario, Canada. Subsequently, from the same four critical care units, we prospectively collected remnant urine specimens containing E. coli or Klebsiella spp. as the dominant Gram-negative in culture and sequenced them using the Oxford Nanopore MK1c. RASE was then used to predict multilocus sequence type (MLST) and antibiotic susceptibility phenotype. The study was approved by the Ottawa Hospital Science Network (#20200108–01H), as well as the University Health Network (#20–5677) and the Sinai Health (20–0161-E) Research Ethics Boards in Toronto, with a waiver of consent from all institutions.

E.coli and Klebsiella spp. isolates from clinical specimens (from any anatomical source) originating from critically ill patients were collected between August 2020 and March 2021 from the four participating Ontario hospitals and stored at −70°C until nucleic acid extraction. Draft whole genome sequences were generated using short-read approaches (Illumina or MGI), and clinically derived antibiotic susceptibility phenotypes were collected as meta-data. Assembled sequences along with antibiotic susceptibility phenotype and genotypic multi-locus sequence typing (MLST) (23, 24, 42) were used to generate the RASE k-mer reference databases (18). For each of E. coli and Klebsiella spp., we constructed a regional database (combination of Toronto and Ottawa isolates). We primarily evaluated the performance of prediction for samples collected from both locations against these regional databases. However, although not the main focus of our investigation, we also performed additional analyses (described below) evaluating the performance of local databases (i.e., using reference databases from the same city to make predictions) and the use of an international (European) database.

Between May 2021 and September 2022, remnant urine specimens from critical care patients submitted to the microbiology laboratory for bacterial culture that were growing a single Gram-negative morphotype of interest (E.coli or Klebsiella spp.) in clinically significant amounts, regardless of the Gram-positive growth, were retrieved and stored at −70°C until nanopore sequencing. Additionally, the Gram-negative bacterial isolates corresponding to the primary specimens of interest were stored at −70°C, and whole genome sequencing was subsequently performed using short-read approaches (Illumina or MGI), and clinically derived antibiotic susceptibility phenotypes were collected as meta-data.

For both reference isolates and uropathogen isolates from primary urine specimens, pathogens were identified to the species level by MALDI-TOF. Antibiotic susceptibility testing was performed as per standard clinical evaluation at their source hospital laboratory. Antibiotic susceptibility testing was performed using a combination of disc diffusion testing, and commercial MIC methods (BD Phoenix M50 [Ottawa], and Vitek 2 [Toronto]) and throughout this period 2020 CLSI breakpoints were used.

Stored remnant urine samples were thawed, and mechanical host cell depletion was performed, followed by DNA extraction and Qubit quantitation. Libraries were prepared using ONT’s Rapid PCR Barcoding Kit with multiplexing. Nanopore metagenomic sequencing was performed on the ONT MinION MK1c sequencer using R9.4.1 flow cells with 11 specimens and a process negative control (12 samples total), for a 24-h run time. Basecalling and subsequent demultiplexing were automatically done using the onboard base calling software guppy (6.4.6). See Fig. 3 for an overview of the method.

After sequencing was completed, de-multiplexed raw read files (fastqs) were selected, and read quality measures output by the ONT software were reviewed. The fastq files containing the first 4000 MinION-generated reads were analyzed using a k-mer-based taxonomic classification system, Kraken2 (version 2.1.3), with standard RefSeq database (k2_standard_20230605) (43), and only prokaryotic reads were retained for storage and analysis. The Kraken2 generated classification report was used to identify the dominant bacterial pathogen in the sample, which was then paired with KrakenTools (44) to filter for pathogen-specific reads, which were used to generate these data. The classification reports also informed selection of the appropriate RASE reference database, which was then used to run the RASE pipeline (version 0.1.0.0; please see Supplementary for detailed usage of RASE). All analyses were run with the Mac OS X operating system.

Prior to defining the MLST and susceptibility predictions made by RASE, we needed to define the point at which the prediction appeared stable. The lineage score (LS) was developed as a potential indicator of lineage prediction confidence, and thus, we chose this to establish a stable prediction (18). Specifically, we defined stability as the point where the LS did not fluctuate more than a magnitude of 0.1 for 100 consecutive reads. The overall read count assigned at the initial time of sequencing was used, which includes both unfiltered and filtered reads, and therefore would facilitate the establishment of a typical range of reads required to achieve a stable prediction that could be applied prospectively.

Antibiotic susceptibility predictions were evaluated for amoxicillin-clavulanic acid, cefazolin, ceftriaxone, ciprofloxacin, gentamicin, meropenem, nitrofurantoin, piperacillin-tazobactam, and trimethoprim-sulfamethoxazole. Standard clinical phenotypic testing of the urine specimen isolate was used as the outcome and was classified as either susceptible (S) or non-susceptible (NS) (which includes both resistant and intermediate susceptibility results). RASE-predicted susceptibility results were directly compared with the clinical phenotype result for the relevant antibiotics for each sample. Test characteristics including sensitivity, specificity, prevalence, positive predictive value (PPV), and negative predictive values (NPV) of susceptibility were calculated (see Supplementary Materials and Methods for further details). PPV was used to indicate the probability of susceptibility if RASE predicted susceptible. 1-NPV was used to generate the probability of susceptibility if RASE predicted non-susceptible. Improvements to susceptible and non-susceptible predictions are indicated by moving these values right and left of the pre-test probability, respectively. We compared these values with previously established thresholds for the probability of susceptibility found to be acceptable to guide treatment in patients with low- to moderate-severity infection (80%) and high-severity infection (90%) (10). These values were compiled for various combinations of antibiotics and organism types, and 95% confidence intervals of test characteristics were also generated. Final values were then visualized using ggplot2 (45), using R version 4.3.0 (46). We generated Bayesian figures of the prevalence, PPV, and 1-NPV values for each of the antibiotics of interest for each organism type (22).

Although the aim of our study was to evaluate regional reference databases for MLST and antibiotic susceptibility phenotype prediction, we also evaluated the predictive performance of local (city-specific) databases and a larger international collection (Supplementary Methods) for predicting MLST and antibiotic susceptibility phenotype. For the latter, we used EuSCAPE isolates, which were collected in hospitals across Europe for the purpose of AMR surveillance (47, 48).

See Supplementary Methods for full details on all described methods.