Benchmarking taxonomic classifiers with Illumina and Nanopore sequence data for clinical metagenomic diagnostic applications

Biases from Oxford Nanopore library preparation kits and their effects on microbiome and genome analysis

BACKGROUND

Oxford Nanopore sequencing is a long-read sequencing technology that does not rely on a polymerase to generate sequence data. Sequencing library preparation methods used in Oxford Nanopore sequencing rely on the addition of a motor protein bound to an adapter sequence, which is added either using ligation-based methods (ligation sequencing kit), or transposase-based methods (rapid sequencing kit). However, these methods have enzymatic steps that may be susceptible to motif bias, including the underrepresentation of adenine-thymine (AT) sequences due to ligation and biases from transposases. This study aimed to compare the recognition motif and relative interaction frequencies of these library preparation methods and assess their effects on relative sequencing coverage, microbiome, and methylation profiles. The impacts of DNA extraction kits and basecalling models on microbiome analysis were also investigated.

RESULTS

By using sequencing data generated by the ligation and rapid library kits, we identified the recognition motif (5'-TATGA-3') consistent with MuA transposase in the rapid kit and low frequencies of AT in the sequence terminus of the ligation kit. The rapid kit showed reduced yield in regions with 40-70% guanine-cytosine (GC) contents, while the ligation kit showed relatively even coverage distribution in areas with various GC contents. Due to longer reads, ligation kits showed increased taxonomic classification efficiency compared to the rapid protocols. Rumen microbial profile at different taxonomic levels and mock community profile showed significant variation due to the library preparation method used. The ligation kit outperformed the rapid kit in subsequent bacterial DNA methylation statistics, although there were no significant differences.

CONCLUSIONS

Our findings indicated that careful and consistent library preparation method selection is essential for quantitative methods such as bovine-related microbiome analysis due to the systematic bias induced by the enzymatic reactions in Oxford Nanopore library preparation.

The use of metagenomics to enhance diagnosis of encephalitis

INTRODUCTION

Encephalitis has a broad etiology, including infectious and auto-immune causes. In infectious encephalitis, the breadth of causative organisms results in incomplete testing and low diagnostic yields.Metagenomics sequences all DNA and RNA allowing untargeted detection of all organisms in a single specimen; this is of particular use in diagnosis of encephalitis with a broad etiology.

AREAS COVERED

We review the literature and discuss metagenomics workflows, host depletion and pathogen enrichment methods, bioinformatics analysis and potential analysis of the host transcriptome to aid diagnosis. We discuss the clinical use of metagenomics for diagnosis of neurological infection including time to result, cost, quality assurance, patient cohorts in whom metagenomics adds the most value, recommended specimen types, limitations and review published cases in which metagenomics has been used to diagnose encephalitis.

EXPERT OPINION

There is good evidence for the utility of metagenomics to diagnose infection in encephalitis. Due to infections with rare, unexpected or novel pathogens, metagenomics adds most value to diagnosis in immunocompromised patients and the greatest diagnostic yield is in brain biopsies. Technical advances are needed to reduce the complexity, cost and time to result which will enable wider adoption in clinical laboratories and use as a first-line test.

Evaluation of the taxonomic classification tools and visualizers for metagenomic analysis using the Oxford nanopore sequence database

Microbial metagenomic identification is generally attributed to the specificity and type of the bioinformatic tools, including classifiers and visualizers. In this study, the performance of two major classifiers, Centrifuge and Kraken2, and two visualizers (Recentrifuge and Krona) has been thoroughly investigated for their efficiency in the identification of the microorganisms using the Whole-Genome Sequence (WGS) database and four targeted databases including NCBI, Silva, Greengenes, and Ribosomal Database Project (RDP). Two standard DNA metagenomic library replicates, Zymo and Zymo-1, were used as quality control. Results showed that Centrifuge gave a higher percentage of Pseudomonas aeruginosa, Escherichia coli, and Salmonella enterica identification than Kraken2. Compared to Recentrifuge, Kraken2 was more accurate in identifying Staphylococcus aureus, Listeria monocytogenes, Bacillus subtilis, and Cryptococcus neoformans. The results of the rest of the detected microorganisms were generally consistent with the two classifiers. Regarding visualizers, both Recentrifuge and Krona provided similar results regarding the abundance of each microbial species regardless of the classifier used. The differences in results between the two mentioned classifiers may be attributed to the specific algorithms each method uses and the sequencing depth. Centrifuge uses a read mapping approach, while Kraken2 uses a k-mer-based system to classify the sequencing reads into taxonomic groups. In conclusion, both Centrifuge and Kraken2 are effective tools for microbial classification. However, the choice of classifier can influence the accuracy of microbial classification and, therefore, should be made carefully, depending on the desired application, even when the same reference database is used.

Anellovirus abundance as an indicator for viral metagenomic classifier utility in plasma samples

BACKGROUND

Viral metagenomics has expanded significantly in recent years due to advancements in next-generation sequencing, establishing it as the leading method for identifying emerging viruses. A crucial step in metagenomics is taxonomic classification, where sequence data is assigned to specific taxa, thereby enabling the characterization of species composition within a sample. Various taxonomic classifiers have been developed in recent years, each employing distinct classification approaches that produce varying results and abundance profiles, even when analyzing the same sample.

METHODS

In this study, we propose using the identification of Torque Teno Viruses (TTVs), from the Anelloviridae family, as indicators to evaluate the performance of four short-read-based metagenomic classifiers: Kraken2, Kaiju, CLARK and DIAMOND, when evaluating human plasma samples.

RESULTS

Our results show that each classifier assigns TTV species at different abundance levels, potentially influencing the interpretation of diversity within samples. Specifically, nucleotide-based classifiers tend to detect a broader range of TTV species, indicating higher sensitivity, while amino acid-based classifiers like DIAMOND and CLARK display lower abundance indices. Interestingly, despite employing different algorithms and data types (protein-based vs. nucleotide-based), Kaiju and Kraken2 performed similarly.

CONCLUSION

Our study underscores the critical impact of classifier selection on diversity indices in metagenomic analyses. Kaiju effectively assigned a wide variety of TTV species, demonstrating it did not require a high volume of reads to capture diversity. Nucleotide-based classifiers like CLARK and Kraken2 showed superior sensitivity, which is valuable for detecting emerging or rare viruses. At the same time, protein-based approaches such as DIAMOND and Kaiju proved robust for identifying known species with low variability.

Impact of database choice and confidence score on the performance of taxonomic classification using Kraken2

Accurate taxonomic classification is essential to understanding microbial diversity and function through metagenomic sequencing. However, this task is complicated by the vast variety of microbial genomes and the computational limitations of bioinformatics tools. The aim of this study was to evaluate the impact of reference database selection and confidence score (CS) settings on the performance of Kraken2, a widely used k-mer-based metagenomic classifier. In this study, we generated simulated metagenomic datasets to systematically evaluate how the choice of reference databases, from the compact Minikraken v1 to the expansive nt- and GTDB r202, and different CS (from 0 to 1.0) affect the key performance metrics of Kraken2. These metrics include classification rate, precision, recall, F1 score, and accuracy of true versus calculated bacterial abundance estimation. Our results show that higher CS, which increases the rigor of taxonomic classification by requiring greater k-mer agreement, generally decreases the classification rate. This effect is particularly pronounced for smaller databases such as Minikraken and Standard-16, where no reads could be classified when the CS was above 0.4. In contrast, for larger databases such as Standard, nt and GTDB r202, precision and F1 scores improved significantly with increasing CS, highlighting their robustness to stringent conditions. Recovery rates were mostly stable, indicating consistent detection of species under different CS settings. Crucially, the results show that a comprehensive reference database combined with a moderate CS (0.2 or 0.4) significantly improves classification accuracy and sensitivity. This finding underscores the need for careful selection of database and CS parameters tailored to specific scientific questions and available computational resources to optimize the results of metagenomic analyses.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42994-024-00178-0.

Evaluation of DNA extraction kits for long-read shotgun metagenomics using Oxford Nanopore sequencing for rapid taxonomic and antimicrobial resistance detection

During a bacterial infection or colonization, the detection of antimicrobial resistance (AMR) is critical, but slow due to culture-based approaches for clinical and screening samples. Culture-based phenotypic AMR detection and confirmation require up to 72 hours (h) or even weeks for slow-growing bacteria. Direct shotgun metagenomics by long-read sequencing using Oxford Nanopore Technologies (ONT) may reduce the time for bacterial species and AMR gene identification. However, screening swabs for metagenomics is complex due to the range of Gram-negative and -positive bacteria, diverse AMR genes, and host DNA present in the samples. Therefore, DNA extraction is a critical initial step. We aimed to compare the performance of different DNA extraction protocols for ONT applications to reliably identify species and AMR genes using a shotgun long-read metagenomic approach. We included three different sample types: ZymoBIOMICS Microbial Community Standard, an in-house mock community of ESKAPE pathogens including Enterococcus faecium, Staphylococcus aureus, Klebsiella pneumoniae, Acinetobacter baumannii, Pseudomonas aeruginosa, and Escherichia coli (ESKAPE Mock), and anonymized clinical swab samples. We processed all sample types with four different DNA extraction kits utilizing different lysis (enzymatic vs. mechanical) and purification (spin-column vs. magnetic beads) methods. We used kits from Qiagen (QIAamp DNA Mini and QIAamp PowerFecal Pro DNA) and Promega (Maxwell RSC Cultured Cells and Maxwell RSC Buccal Swab DNA). After extraction, samples were subject to the Rapid Barcoding Kit (RBK004) for library preparation followed by sequencing on the GridION with R9.4.1 flow cells. The fast5 files were base called to fastq files using Guppy in High Accuracy (HAC) mode with the inbuilt MinKNOW software. Raw read quality was assessed using NanoPlot and human reads were removed using Minimap2 alignment against the Hg38 genome. Taxonomy identification was performed on the raw reads using Kraken2 and on assembled contigs using Minimap2. The AMR genes were identified using Minimap2 with alignment against the CARD database on both the raw reads and assembled contigs. We identified all bacterial species present in the Zymo Mock Community (8/8) and ESKAPE Mock (6/6) with Qiagen PowerFecal Pro DNA kit (chemical and mechanical lysis) at read and assembly levels. Enzymatic lysis retrieved fewer aligned bases for the Gram-positive species (Staphylococcus aureus and Enterococcus faecium) from the ESKAPE Mock on the assembly level compared to the mechanical lysis. We detected the AMR genes from Gram-negative and -positive species in the ESKAPE Mock with the QIAamp PowerFecal Pro DNA kit on reads level with a maximum median time of 1.9 h of sequencing. Long-read metagenomics with ONT may reduce the turnaround time in screening for AMR genes. Currently, the QIAamp PowerFecal Pro DNA kit (chemical and mechanical lysis) for DNA extraction along with the Rapid Barcoding Kit for the ONT sequencing captured the best taxonomy and AMR identification for our specific use case.

Evaluating metagenomics and targeted approaches for diagnosis and surveillance of viruses

BACKGROUND

Metagenomics is a powerful approach for the detection of unknown and novel pathogens. Workflows based on Illumina short-read sequencing are becoming established in diagnostic laboratories. However, high sequencing depth requirements, long turnaround times, and limited sensitivity hinder broader adoption. We investigated whether we could overcome these limitations using protocols based on untargeted sequencing with Oxford Nanopore Technologies (ONT), which offers real-time data acquisition and analysis, or a targeted panel approach, which allows the selective sequencing of known pathogens and could improve sensitivity.

METHODS

We evaluated detection of viruses with readily available untargeted metagenomic workflows using Illumina and ONT, and an Illumina-based enrichment approach using the Twist Bioscience Comprehensive Viral Research Panel (CVRP), which targets 3153 viruses. We tested samples consisting of a dilution series of a six-virus mock community in a human DNA/RNA background, designed to resemble clinical specimens with low microbial abundance and high host content. Protocols were designed to retain the host transcriptome, since this could help confirm the absence of infectious agents. We further compared the performance of commonly used taxonomic classifiers.

RESULTS

Capture with the Twist CVRP increased sensitivity by at least 10-100-fold over untargeted sequencing, making it suitable for the detection of low viral loads (60 genome copies per ml (gc/ml)), but additional methods may be needed in a diagnostic setting to detect untargeted organisms. While untargeted ONT had good sensitivity at high viral loads (60,000 gc/ml), at lower viral loads (600-6000 gc/ml), longer and more costly sequencing runs would be required to achieve sensitivities comparable to the untargeted Illumina protocol. Untargeted ONT provided better specificity than untargeted Illumina sequencing. However, the application of robust thresholds standardized results between taxonomic classifiers. Host gene expression analysis is optimal with untargeted Illumina sequencing but possible with both the CVRP and ONT.

CONCLUSIONS

Metagenomics has the potential to become standard-of-care in diagnostics and is a powerful tool for the discovery of emerging pathogens. Untargeted Illumina and ONT metagenomics and capture with the Twist CVRP have different advantages with respect to sensitivity, specificity, turnaround time and cost, and the optimal method will depend on the clinical context.

Benchmarking bacterial taxonomic classification using nanopore metagenomics data of several mock communities

Taxonomic classification is crucial in identifying organisms within diverse microbial communities when using metagenomics shotgun sequencing. While second-generation Illumina sequencing still dominates, third-generation nanopore sequencing promises improved classification through longer reads. However, extensive benchmarking studies on nanopore data are lacking. We systematically evaluated performance of bacterial taxonomic classification for metagenomics nanopore sequencing data for several commonly used classifiers, using standardized reference sequence databases, on the largest collection of publicly available data for defined mock communities thus far (nine samples), representing different research domains and application scopes. Our results categorize classifiers into three categories: low precision/high recall; medium precision/medium recall, and high precision/medium recall. Most fall into the first group, although precision can be improved without excessively penalizing recall with suitable abundance filtering. No definitive 'best' classifier emerges, and classifier selection depends on application scope and practical requirements. Although few classifiers designed for long reads exist, they generally exhibit better performance. Our comprehensive benchmarking provides concrete recommendations, supported by publicly available code for reassessment and fine-tuning by other scientists.

An in-depth evaluation of metagenomic classifiers for soil microbiomes

BACKGROUND

Recent endeavours in metagenomics, exemplified by projects such as the human microbiome project and TARA Oceans, have illuminated the complexities of microbial biomes. A robust bioinformatic pipeline and meticulous evaluation of their methodology have contributed to the success of these projects. The soil environment, however, with its unique challenges, requires a specialized methodological exploration to maximize microbial insights. A notable limitation in soil microbiome studies is the dearth of soil-specific reference databases available to classifiers that emulate the complexity of soil communities. There is also a lack of in-vitro mock communities derived from soil strains that can be assessed for taxonomic classification accuracy.

RESULTS

In this study, we generated a custom in-silico mock community containing microbial genomes commonly observed in the soil microbiome. Using this mock community, we simulated shotgun sequencing data to evaluate the performance of three leading metagenomic classifiers: Kraken2 (supplemented with Bracken, using a custom database derived from GTDB-TK genomes along with its own default database), Kaiju, and MetaPhlAn, utilizing their respective default databases for a robust analysis. Our results highlight the importance of optimizing taxonomic classification parameters, database selection, as well as analysing trimmed reads and contigs. Our study showed that classifiers tailored to the specific taxa present in our samples led to fewer errors compared to broader databases including microbial eukaryotes, protozoa, or human genomes, highlighting the effectiveness of targeted taxonomic classification. Notably, an optimal classifier performance was achieved when applying a relative abundance threshold of 0.001% or 0.005%. The Kraken2 supplemented with bracken, with a custom database demonstrated superior precision, sensitivity, F1 score, and overall sequence classification. Using a custom database, this classifier classified 99% of in-silico reads and 58% of real-world soil shotgun reads, with the latter identifying previously overlooked phyla using a custom database.

CONCLUSION

This study underscores the potential advantages of in-silico methodological optimization in metagenomic analyses, especially when deciphering the complexities of soil microbiomes. We demonstrate that the choice of classifier and database significantly impacts microbial taxonomic profiling. Our findings suggest that employing Kraken2 with Bracken, coupled with a custom database of GTDB-TK genomes and fungal genomes at a relative abundance threshold of 0.001% provides optimal accuracy in soil shotgun metagenome analysis.

Interpretation of vaginal metagenomic characteristics in different types of vaginitis

Although vaginitis is closely related to vaginal microecology in females, the precise composition and functional potential of different types of vaginitis remain unclear. Here, metagenomic sequencing was applied to analyze the vaginal flora in patients with various forms of vaginitis, including cases with a clue cell proportion ranging from 1% to 20% (Clue1_20), bacterial vaginitis (BV), vulvovaginal candidiasis (VVC), and BV combined with VVC (VVC_BV). Our results identified Prevotella as an important biomarker between BV and Clue1_20. Moreover, a gradual decrease was observed in the relative abundance of shikimic acid metabolism associated with bacteria producing indole as well as a decline in the abundance of Gardnerella vaginalis in patients with BV, Clue1_20, and healthy women. Interestingly, the vaginal flora of patients in the VVC_BV group exhibited structural similarities to that of the VVC group, and its potentially functional characteristics resembled those of the BV and VVC groups. Finally, Lactobacillus crispatus was found in high abundance in healthy samples, greatly contributing to the stability of the vaginal environment. For the further study of L. crispatus, we isolated five strains of L. crispatus from healthy samples and evaluated their capacity to inhibit G. vaginalis biofilms and produce lactic acid in vitro to select the potential probiotic candidate for improving vaginitis in future clinical studies. Overall, we successfully identified bacterial biomarkers of different vaginitis and characterized the dynamic shifts in vaginal flora between patients with BV and healthy females. This research advances our understanding and holds great promise in enhancing clinical approaches for the treatment of vaginitis.

IMPORTANCE

Vaginitis is one of the most common gynecological diseases, mostly caused by infections of pathogens such as Candida albicans and Gardnerella vaginalis. In recent years, it has been found that the stability of the vaginal flora plays an important role in vaginitis. Furthermore, the abundant Lactobacillus-producing rich lactic acid in the vagina provides a healthy acidic environment such as Lactobacillus crispatus. The metabolites of Lactobacillus can inhibit the colonization of pathogens. Here, we collected the vaginal samples of patients with bacterial vaginitis (BV), vulvovaginal candidiasis (VVC), and BV combined with VVC to discover the differences and relationships among the different kinds of vaginitis by metagenomic sequencing. Furthermore, because of the importance of L. crispatus in promoting vaginal health, we isolated multiple strains from vaginal samples of healthy females and chose the most promising strain with potential probiotic benefits to provide clinical implications for treatment strategies.

Enhancing Clinical Utility: Utilization of International Standards and Guidelines for Metagenomic Sequencing in Infectious Disease Diagnosis

Metagenomic sequencing has emerged as a transformative tool in infectious disease diagnosis, offering a comprehensive and unbiased approach to pathogen detection. Leveraging international standards and guidelines is essential for ensuring the quality and reliability of metagenomic sequencing in clinical practice. This review explores the implications of international standards and guidelines for the application of metagenomic sequencing in infectious disease diagnosis. By adhering to established standards, such as those outlined by regulatory bodies and expert consensus, healthcare providers can enhance the accuracy and clinical utility of metagenomic sequencing. The integration of international standards and guidelines into metagenomic sequencing workflows can streamline diagnostic processes, improve pathogen identification, and optimize patient care. Strategies in implementing these standards for infectious disease diagnosis using metagenomic sequencing are discussed, highlighting the importance of standardized approaches in advancing precision infectious disease diagnosis initiatives.

Unveiling the microbiome during post-partum uterine infection: a deep shotgun sequencing approach to characterize the dairy cow uterine microbiome

BACKGROUND

The goal of this study was to assess the microbial ecology and diversity present in the uterus of post-partum dairy cows with and without metritis from 24 commercial California dairy farms using shotgun metagenomics. A set subset of 95 intrauterine swab samples, taken from a larger selection of 307 individual cow samples previously collected, were examined for α and β diversity and differential abundance associated with metritis. Cows within 21 days post-partum were categorized into one of three clinical groups during sample collection: control (CT, n = 32), defined as cows with either no vaginal discharge or a clear, non-purulent mucus vaginal discharge; metritis (MET, n = 33), defined as a cow with watery, red or brown colored, and fetid vaginal discharge; and purulent discharge cows (PUS, n = 31), defined as a non-fetid purulent or mucopurulent vaginal discharge.

RESULTS

All three clinical groups (CT, MET, and PUS) were highly diverse, with the top 12 most abundant genera accounting for 10.3%, 8.8%, and 10.1% of mean relative abundance, respectively. The α diversity indices revealed a lower diversity from samples collected from MET and PUS when compared to CT cows. PERMANOVA statistical testing revealed a significant difference (P adjusted < 0.01) in the diversity of genera between CT and MET samples (R2 = 0.112, P = 0.003) and a non-significant difference between MET and PUS samples (R2 = 0.036, P = 0.046). ANCOM-BC analysis revealed that from the top 12 most abundant genera, seven genera were increased in the natural log fold change (LFC) of abundance in MET when compared to CT samples: Bacteroides, Clostridium, Fusobacterium, Phocaeicola, Porphyromonas, Prevotella, and Streptococcus. Two genera, Dietzia and Microbacterium, were decreased in natural LFC of abundance when comparing MET (regardless of treatment) and CT, while no changes in natural LFC of abundance were observed for Escherichia, Histophilus, and Trueperella.

CONCLUSIONS

The results presented here, are the current deepest shotgun metagenomic analyses conducted on the bovine uterine microbiome to date (mean of 256,425 genus-level reads per sample). Our findings support that uterine samples from cows without metritis (CT) had increased α-diversity but decreased β-diversity when compared to metritis or PUS cows, characteristic of dysbiosis. In summary, our findings highlight that MET cows have an increased abundance of Bacteroides, Porphyromonas, and Fusobacterium when compared to CT and PUS, and support the need for further studies to better understand their potential causal role in metritis pathogenesis.

Rapid metagenomic sequencing for diagnosis and antimicrobial sensitivity prediction of canine bacterial infections

Antimicrobial resistance is a major threat to human and animal health. There is an urgent need to ensure that antimicrobials are used appropriately to limit the emergence and impact of resistance. In the human and veterinary healthcare setting, traditional culture and antimicrobial sensitivity testing typically requires 48-72 h to identify appropriate antibiotics for treatment. In the meantime, broad-spectrum antimicrobials are often used, which may be ineffective or impact non-target commensal bacteria. Here, we present a rapid, culture-free, diagnostics pipeline, involving metagenomic nanopore sequencing directly from clinical urine and skin samples of dogs. We have planned this pipeline to be versatile and easily implementable in a clinical setting, with the potential for future adaptation to different sample types and animals. Using our approach, we can identify the bacterial pathogen present within 5 h, in some cases detecting species which are difficult to culture. For urine samples, we can predict antibiotic sensitivity with up to 95 % accuracy. Skin swabs usually have lower bacterial abundance and higher host DNA, confounding antibiotic sensitivity prediction; an additional host depletion step will likely be required during the processing of these, and other types of samples with high levels of host cell contamination. In summary, our pipeline represents an important step towards the design of individually tailored veterinary treatment plans on the same day as presentation, facilitating the effective use of antibiotics and promoting better antimicrobial stewardship.