INGOT-DR: an interpretable classifier for predicting drug resistance in M. tuberculosis

Assessing computational predictions of antimicrobial resistance phenotypes from microbial genomes

The advent of rapid whole-genome sequencing has created new opportunities for computational prediction of antimicrobial resistance (AMR) phenotypes from genomic data. Both rule-based and machine learning (ML) approaches have been explored for this task, but systematic benchmarking is still needed. Here, we evaluated four state-of-the-art ML methods (Kover, PhenotypeSeeker, Seq2Geno2Pheno and Aytan-Aktug), an ML baseline and the rule-based ResFinder by training and testing each of them across 78 species-antibiotic datasets, using a rigorous benchmarking workflow that integrates three evaluation approaches, each paired with three distinct sample splitting methods. Our analysis revealed considerable variation in the performance across techniques and datasets. Whereas ML methods generally excelled for closely related strains, ResFinder excelled for handling divergent genomes. Overall, Kover most frequently ranked top among the ML approaches, followed by PhenotypeSeeker and Seq2Geno2Pheno. AMR phenotypes for antibiotic classes such as macrolides and sulfonamides were predicted with the highest accuracies. The quality of predictions varied substantially across species-antibiotic combinations, particularly for beta-lactams; across species, resistance phenotyping of the beta-lactams compound, aztreonam, amoxicillin/clavulanic acid, cefoxitin, ceftazidime and piperacillin/tazobactam, alongside tetracyclines demonstrated more variable performance than the other benchmarked antibiotics. By organism, Campylobacter jejuni and Enterococcus faecium phenotypes were more robustly predicted than those of Escherichia coli, Staphylococcus aureus, Salmonella enterica, Neisseria gonorrhoeae, Klebsiella pneumoniae, Pseudomonas aeruginosa, Acinetobacter baumannii, Streptococcus pneumoniae and Mycobacterium tuberculosis. In addition, our study provides software recommendations for each species-antibiotic combination. It furthermore highlights the need for optimization for robust clinical applications, particularly for strains that diverge substantially from those used for training.

Machine Learning of the Whole Genome Sequence of Mycobacterium tuberculosis: A Scoping PRISMA-Based Review

Tuberculosis (TB) remains one of the most significant global health problems, posing a significant challenge to public health systems worldwide. However, diagnosing drug-resistant tuberculosis (DR-TB) has become increasingly challenging due to the rising number of multidrug-resistant (MDR-TB) cases, despite the development of new TB diagnostic tools. Even the World Health Organization-recommended methods such as Xpert MTB/XDR or Truenat are unable to detect all the Mycobacterium tuberculosis genome mutations associated with drug resistance. While Whole Genome Sequencing offers a more precise DR profile, the lack of user-friendly bioinformatics analysis applications hinders its widespread use. This review focuses on exploring various artificial intelligence models for predicting DR-TB profiles, analyzing relevant English-language articles using the PRISMA methodology through the Covidence platform. Our findings indicate that an Artificial Neural Network is the most commonly employed method, with non-statistical dimensionality reduction techniques preferred over traditional statistical approaches such as Principal Component Analysis or t-distributed Stochastic Neighbor Embedding.

Machine learning models for Neisseria gonorrhoeae antimicrobial susceptibility tests

Neisseria gonorrhoeae is an urgent public health threat due to the emergence of antibiotic resistance. As most isolates in the United States are susceptible to at least one antibiotic, rapid molecular antimicrobial susceptibility tests (ASTs) would offer the opportunity to tailor antibiotic therapy, thereby expanding treatment options. With genome sequence and antibiotic resistance phenotype data for nearly 20,000 clinical N. gonorrhoeae isolates now available, there is an opportunity to use statistical methods to develop sequence-based diagnostics that predict antibiotic susceptibility from genotype. N. gonorrhoeae, therefore, provides a useful example illustrating how to apply machine learning models to aid in the design of sequence-based ASTs. We present an overview of this framework, which begins with establishing the assay technology, the performance criteria, the population in which the diagnostic will be used, and the clinical goals, and extends to the choices that must be made to arrive at a set of features with the desired properties for predicting susceptibility phenotype from genotype. While we focus on the example of N. gonorrhoeae, the framework generalizes to other organisms for which large-scale genotype and antibiotic resistance data can be combined to aid in diagnostics development.

A convolutional neural network highlights mutations relevant to antimicrobial resistance in Mycobacterium tuberculosis

Long diagnostic wait times hinder international efforts to address antibiotic resistance in M. tuberculosis. Pathogen whole genome sequencing, coupled with statistical and machine learning models, offers a promising solution. However, generalizability and clinical adoption have been limited by a lack of interpretability, especially in deep learning methods. Here, we present two deep convolutional neural networks that predict antibiotic resistance phenotypes of M. tuberculosis isolates: a multi-drug CNN (MD-CNN), that predicts resistance to 13 antibiotics based on 18 genomic loci, with AUCs 82.6-99.5% and higher sensitivity than state-of-the-art methods; and a set of 13 single-drug CNNs (SD-CNN) with AUCs 80.1-97.1% and higher specificity than the previous state-of-the-art. Using saliency methods to evaluate the contribution of input sequence features to the SD-CNN predictions, we identify 18 sites in the genome not previously associated with resistance. The CNN models permit functional variant discovery, biologically meaningful interpretation, and clinical applicability.

External validation of WGS-based antimicrobial susceptibility prediction tools, KOVER-AMR and ResFinder 4.1 for Escherichia coli clinical isolates

OBJECTIVE

To externally validate whole genome sequence-antimicrobial susceptibility testing (WGS-AST) phenotype prediction tools KOVER-AMR and ResFinder 4.1 for Escherichia coli clinical isolates from Dutch routine care.

METHODS

A random sample of 234 E. coli, and 283 3^rd generation cephalosporin-resistant E. coli isolates from urine and blood were collected (2014-17). Culture-AST was performed using VITEK 2 and BD Phoenix. Sequences were used as input for KOVER-AMR-SCM, KOVER-AMR-CART and ResFinder 4.1. The concordance, major error rate (MER), and very major error rate (VMER) were calculated, with subsequent comparison to U.S. Food and Drug Administration (FDA) criteria (MER ≤3%, and VMER with a 95% confidence interval (CI) ≤1.5%-≤7.5%).

RESULTS

ResFinder 4.1 performed better than KOVER-AMR-models, however, neither tools achieved overall (V)MERs below FDA criteria. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1, MER (cumulative all antimicrobials) were: 5.1% (4.4-5.9), 4.3% (3.6-5.0), and 5.1% (4.5-5.8), respectively. MERs ≤3% were achieved for 6 (SCM) and 5 (CART) of the 11 tested antimicrobials for KOVER-AMR-models, and for 9/13 antimicrobials tested with ResFinder 4.1. KOVER-AMR-SCM, KOVER-AMR-CART, and ResFinder 4.1, cumulative VMERs were: 26% (24-28), 29% (27-31), and 11% (9.2-12). VMERs with a 95%CI ≤1.5-≤7.5 were only achieved for 4/13 tested antimicrobials with ResFinder 4.1.

CONCLUSION

In this study, WGS-AST phenotype prediction tools, KOVER-AMR and ResFinder 4.1, did not meet the FDA criteria needed for clinical diagnostic use in 517 E. coli clinical isolates from Dutch routine care. The tested tools should be further improved before they can be used for clinical decision making.

The 2021 WHO catalogue of Mycobacterium tuberculosis complex mutations associated with drug resistance: A genotypic analysis

Background

Molecular diagnostics are considered the most promising route to achieving rapid, universal drug susceptibility testing for Mycobacterium tuberculosiscomplex (MTBC). We aimed to generate a WHO endorsed catalogue of mutations to serve as a global standard for interpreting molecular information for drug resistance prediction.

Methods

A candidate gene approach was used to identify mutations as associated with resistance, or consistent with susceptibility, for 13 WHO endorsed anti-tuberculosis drugs. 38,215 MTBC isolates with paired whole-genome sequencing and phenotypic drug susceptibility testing data were amassed from 45 countries. For each mutation, a contingency table of binary phenotypes and presence or absence of the mutation computed positive predictive value, and Fisher's exact tests generated odds ratios and Benjamini-Hochberg corrected p-values. Mutations were graded as Associated with Resistance if present in at least 5 isolates, if the odds ratio was >1 with a statistically significant corrected p-value, and if the lower bound of the 95% confidence interval on the positive predictive value for phenotypic resistance was >25%. A series of expert rules were applied for final confidence grading of each mutation.

Findings

15,667 associations were computed for 13,211 unique mutations linked to one or more drugs. 1,149/15,667 (7·3%) mutations were classified as associated with phenotypic resistance and 107/15,667 (0·7%) were deemed consistent with susceptibility. For rifampicin, isoniazid, ethambutol, fluoroquinolones, and streptomycin, the mutations' pooled sensitivity was >80%. Specificity was over 95% for all drugs except ethionamide (91·4%), moxifloxacin (91·6%) and ethambutol (93·3%). Only two resistance mutations were classified for bedaquiline, delamanid, clofazimine, and linezolid as prevalence of phenotypic resistance was low for these drugs.

Interpretation

This first WHO endorsed catalogue of molecular targets for MTBC drug susceptibility testing provides a global standard for resistance interpretation. Its existence should encourage the implementation of molecular diagnostics by National Tuberculosis Programmes.

Funding

UNITAID, Wellcome, MRC, BMGF.