Proficiency Testing of Virus Diagnostics Based on Bioinformatics Analysis of Simulated In Silico High-Throughput Sequencing Data Sets

INSaFLU-TELEVIR: an open web-based bioinformatics suite for viral metagenomic detection and routine genomic surveillance

BACKGROUND

Implementation of clinical metagenomics and pathogen genomic surveillance can be particularly challenging due to the lack of bioinformatics tools and/or expertise. In order to face this challenge, we have previously developed INSaFLU, a free web-based bioinformatics platform for virus next-generation sequencing data analysis. Here, we considerably expanded its genomic surveillance component and developed a new module (TELEVIR) for metagenomic virus identification.

RESULTS

The routine genomic surveillance component was strengthened with new workflows and functionalities, including (i) a reference-based genome assembly pipeline for Oxford Nanopore technologies (ONT) data; (ii) automated SARS-CoV-2 lineage classification; (iii) Nextclade analysis; (iv) Nextstrain phylogeographic and temporal analysis (SARS-CoV-2, human and avian influenza, monkeypox, respiratory syncytial virus (RSV A/B), as well as a "generic" build for other viruses); and (v) algn2pheno for screening mutations of interest. Both INSaFLU pipelines for reference-based consensus generation (Illumina and ONT) were benchmarked against commonly used command line bioinformatics workflows for SARS-CoV-2, and an INSaFLU snakemake version was released. In parallel, a new module (TELEVIR) for virus detection was developed, after extensive benchmarking of state-of-the-art metagenomics software and following up-to-date recommendations and practices in the field. TELEVIR allows running complex workflows, covering several combinations of steps (e.g., with/without viral enrichment or host depletion), classification software (e.g., Kaiju, Kraken2, Centrifuge, FastViromeExplorer), and databases (RefSeq viral genome, Virosaurus, etc.), while culminating in user- and diagnosis-oriented reports. Finally, to potentiate real-time virus detection during ONT runs, we developed findONTime, a tool aimed at reducing costs and the time between sample reception and diagnosis.

CONCLUSIONS

The accessibility, versatility, and functionality of INSaFLU-TELEVIR are expected to supply public and animal health laboratories and researchers with a user-oriented and pan-viral bioinformatics framework that promotes a strengthened and timely viral metagenomic detection and routine genomics surveillance. INSaFLU-TELEVIR is compatible with Illumina, Ion Torrent, and ONT data and is freely available at https://insaflu.insa.pt/ (online tool) and https://github.com/INSaFLU (code).

Assessing the Quality of Metagenomic Next-Generation Sequencing for Pathogen Detection in Lower Respiratory Infections

BACKGROUND

Laboratory-developed metagenomic next-generation sequencing (mNGS) assays are increasingly being used for the diagnosis of infectious disease. To ensure comparable results and advance the quality control for the mNGS assay, we initiated a large-scale multicenter quality assessment to scrutinize the ability of mNGS to detect pathogens in lower respiratory infections.

METHODS

A reference panel containing artificial microbial communities and real clinical samples was used to assess the performance of 122 laboratories. We comprehensively evaluated the reliability, the source of false-positive and false-negative microbes, as well as the ability to interpret the results.

RESULTS

A wide variety of weighted F1-scores was observed across 122 participants, with a range from 0.20 to 0.97. The majority of false positive microbes (68.56%, 399/582) were introduced from "wet lab." The loss of microbial sequence during wet labs was the chief cause (76.18%, 275/361) of false-negative errors. When the human context is 2 × 105 copies/mL, most DNA and RNA viruses at titers above 104 copies/mL could be detected by >80% of the participants, while >90% of the laboratories could detect bacteria and fungi at titers lower than 103 copies/mL. A total of 10.66% (13/122) to 38.52% (47/122) of the participants could detect the target pathogens but failed to reach a correct etiological diagnosis.

CONCLUSIONS

This study clarified the sources of false-positive and false-negative results and evaluated the performance of interpreting the results. This study was valuable for clinical mNGS laboratories to improve method development, avoid erroneous results being reported, and implement regulatory quality controls in the clinic.

One Health surveillance-A cross-sectoral detection, characterization, and notification of foodborne pathogens

Introduction

Several Proficiency Test (PT) or External Quality Assessment (EQA) schemes are currently available for assessing the ability of laboratories to detect and characterize enteropathogenic bacteria, but they are usually targeting one sector, covering either public health, food safety or animal health. In addition to sector-specific PTs/EQAs for detection, cross-sectoral panels would be useful for assessment of the capacity to detect and characterize foodborne pathogens in a One Health (OH) perspective and further improving food safety and interpretation of cross-sectoral surveillance data. The aims of the study were to assess the cross-sectoral capability of European public health, animal health and food safety laboratories to detect, characterize and notify findings of the foodborne pathogens Campylobacter spp., Salmonella spp. and Yersinia enterocolitica, and to develop recommendations for future cross-sectoral PTs and EQAs within OH. The PT/EQA scheme developed within this study consisted of a test panel of five samples, designed to represent a theoretical outbreak scenario.

Methods

A total of 15 laboratories from animal health, public health and food safety sectors were enrolled in eight countries: Denmark, France, Italy, the Netherlands, Poland, Spain, Sweden, and the United Kingdom. The laboratories analyzed the samples according to the methods used in the laboratory and reported the target organisms at species level, and if applicable, serovar for Salmonella and bioserotype for Yersinia.

Results

All 15 laboratories analyzed the samples for Salmonella, 13 for Campylobacter and 11 for Yersinia. Analytical errors were predominately false negative results. One sample (S. Stockholm and Y. enterocolitica O:3/BT4) with lower concentrations of target organisms was especially challenging, resulting in six out of seven false negative results. These findings were associated with laboratories using smaller sample sizes and not using enrichment methods. Detection of Salmonella was most commonly mandatory to notify within the three sectors in the eight countries participating in the pilot whereas findings of Campylobacter and Y. enterocolitica were notifiable from human samples, but less commonly from animal and food samples.

Discussion

The results of the pilot PT/EQA conducted in this study confirmed the possibility to apply a cross-sectoral approach for assessment of the joint OH capacity to detect and characterize foodborne pathogens.

Performance of Five Metagenomic Classifiers for Virus Pathogen Detection Using Respiratory Samples from a Clinical Cohort

Viral metagenomics is increasingly applied in clinical diagnostic settings for detection of pathogenic viruses. While several benchmarking studies have been published on the use of metagenomic classifiers for abundance and diversity profiling of bacterial populations, studies on the comparative performance of the classifiers for virus pathogen detection are scarce. In this study, metagenomic data sets (n = 88) from a clinical cohort of patients with respiratory complaints were used for comparison of the performance of five taxonomic classifiers: Centrifuge, Clark, Kaiju, Kraken2, and Genome Detective. A total of 1144 positive and negative PCR results for a total of 13 respiratory viruses were used as gold standard. Sensitivity and specificity of these classifiers ranged from 83 to 100% and 90 to 99%, respectively, and was dependent on the classification level and data pre-processing. Exclusion of human reads generally resulted in increased specificity. Normalization of read counts for genome length resulted in a minor effect on overall performance, however it negatively affected the detection of targets with read counts around detection level. Correlation of sequence read counts with PCR Ct-values varied per classifier, data pre-processing (R² range 15.1–63.4%), and per virus, with outliers up to 3 log₁₀ reads magnitude beyond the predicted read count for viruses with high sequence diversity. In this benchmarking study, sensitivity and specificity were within the ranges of use for diagnostic practice when the cut-off for defining a positive result was considered per classifier.

Side-by-Side Comparison of Post-Entry Quarantine and High Throughput Sequencing Methods for Virus and Viroid Diagnosis

Rapid and safe access to new plant genetic stocks is crucial for primary plant industries to remain profitable, sustainable, and internationally competitive. Imported plant species may spend several years in Post Entry Quarantine (PEQ) facilities, undergoing pathogen testing which can impact the ability of plant industries to quickly adapt to new global market opportunities by accessing new varieties. Advances in high throughput sequencing (HTS) technologies provide new opportunities for a broad range of fields, including phytosanitary diagnostics. In this study, we compare the performance of two HTS methods (RNA-Seq and sRNA-Seq) with that of existing PEQ molecular assays in detecting and identifying viruses and viroids from various plant commodities. To analyze the data, we tested several bioinformatics tools which rely on different approaches, including direct-read, de novo, and reference-guided assembly. We implemented VirusReport, a new portable, scalable, and reproducible nextflow pipeline that analyses sRNA datasets to detect and identify viruses and viroids. We raise awareness of the need to evaluate cross-sample contamination when analyzing HTS data routinely and of using methods to mitigate index cross-talk. Overall, our results suggest that sRNA analyzed using VirReport provides opportunities to improve quarantine testing at PEQ by detecting all regulated exotic viruses from imported plants in a single assay.

Multilaboratory assessment of metagenomic next-generation sequencing for unbiased microbe detection

•

We developed a set of well-defined reference materials, which is very beneficial to monitor problems in mNGS workflows and identify optimal protocols.

•

The high interlaboratory variability in the identification and quantitation of microbes indicates that the current mNGS protocols are in urgent need of standardization and optimization.

•

The detection rate of mNGS for low-concentration microbes (less than 10³ cell/ml) is significantly lower than that of microbes with a concentration of 10⁴ cell/ml and higher.

•

Only 56.7% to 83.3% of the laboratories showed a sufficient ability to obtain clear etiological diagnoses for three simulated cases combined with patient information.

•

Addressing laboratory contamination(false positive) is an urgent task.

Introduction

Metagenomic next-generation sequencing (mNGS) assay for detecting infectious agents is now in the stage of being translated into clinical practice. With no approved approaches or guidelines available, laboratories adopt customized mNGS assays to detect clinical samples. However, the accuracy, reliability, and problems of these routinely implemented assays are not clear.

Objectives

To evaluate the performance of 90 mNGS laboratories under routine testing conditions through analyzing identical samples.

Methods

Eleven microbial communities were generated using 15 quantitative microbial suspensions. They were used as reference materials to evaluate the false negatives and false positives of participating mNGS protocols, as well as the ability to distinguish genetically similar organisms and to identify true pathogens from other microbes based on fictitious case reports.

Results

High interlaboratory variability was found in the identification and the quantitative reads per million reads (RPM) values of each microbe in the samples, especially when testing microbes present at low concentrations (1 × 10³ cell/ml or less). 42.2% (38/90) of the laboratories reported unexpected microbes (i.e. false positive problem). Only 56.7% (51/90) to 83.3% (75/90) of the laboratories showed a sufficient ability to obtain clear etiological diagnoses for three simulated cases combined with patient information. The analysis of the performance of mNGS in distinguishing genetically similar organisms in three samples revealed that only 56.6% to 63.0% of the laboratories recovered RPM ratios (RPM_{S. aureus}/RPM_{S. epidermidis}) within the range of a 2-fold change of the initial input ratios (indicating a relatively low level of bias).

Conclusion

The high interlaboratory variability found in both identifying microbes and distinguishing true pathogens emphasizes the urgent need for improving the accuracy and comparability of the results generated across different mNGS laboratories, especially in the detection of low-microbial-biomass samples.

Benchmark of thirteen bioinformatic pipelines for metagenomic virus diagnostics using datasets from clinical samples

INTRODUCTION

Metagenomic sequencing is increasingly being used in clinical settings for difficult to diagnose cases. The performance of viral metagenomic protocols relies to a large extent on the bioinformatic analysis. In this study, the European Society for Clinical Virology (ESCV) Network on NGS (ENNGS) initiated a benchmark of metagenomic pipelines currently used in clinical virological laboratories.

METHODS

Metagenomic datasets from 13 clinical samples from patients with encephalitis or viral respiratory infections characterized by PCR were selected. The datasets were analyzed with 13 different pipelines currently used in virological diagnostic laboratories of participating ENNGS members. The pipelines and classification tools were: Centrifuge, DAMIAN, DIAMOND, DNASTAR, FEVIR, Genome Detective, Jovian, MetaMIC, MetaMix, One Codex, RIEMS, VirMet, and Taxonomer. Performance, characteristics, clinical use, and user-friendliness of these pipelines were analyzed.

RESULTS

Overall, viral pathogens with high loads were detected by all the evaluated metagenomic pipelines. In contrast, lower abundance pathogens and mixed infections were only detected by 3/13 pipelines, namely DNASTAR, FEVIR, and MetaMix. Overall sensitivity ranged from 80% (10/13) to 100% (13/13 datasets). Overall positive predictive value ranged from 71-100%. The majority of the pipelines classified sequences based on nucleotide similarity (8/13), only a minority used amino acid similarity, and 6 of the 13 pipelines assembled sequences de novo. No clear differences in performance were detected that correlated with these classification approaches. Read counts of target viruses varied between the pipelines over a range of 2-3 log, indicating differences in limit of detection.

CONCLUSION

A wide variety of viral metagenomic pipelines is currently used in the participating clinical diagnostic laboratories. Detection of low abundant viral pathogens and mixed infections remains a challenge, implicating the need for standardization and validation of metagenomic analysis for clinical diagnostic use. Future studies should address the selective effects due to the choice of different reference viral databases.

Proficiency testing for bacterial whole genome sequencing in assuring the quality of microbiology diagnostics in clinical and public health laboratories

The adoption of whole genome sequencing (WGS) data over the past decade for pathogen surveillance, and decision-making for infectious diseases has rapidly transformed the landscape of clinical microbiology and public health. However, for successful transition to routine use of these techniques, it is crucial to ensure the WGS data generated meet defined quality standards for pathogen identification, typing, antimicrobial resistance detection and surveillance. Further, the ongoing development of these standards will ensure that the bioinformatic processes are capable of accurately identifying and characterising organisms of interest, and thereby facilitate the integration of WGS into routine clinical and public health laboratory setting. A pilot proficiency testing (PT) program for WGS of infectious agents was developed to facilitate widely applicable standardisation and benchmarking standards for WGS across a range of laboratories. The PT participating laboratories were required to generate WGS data from two bacterial isolates, and submit the raw data for independent bioinformatics analysis, as well as analyse the data with their own processes and answer relevant questions about the data. Overall, laboratories used a diverse range of bioinformatics tools and could generate and analyse high-quality data, either meeting or exceeding the minimum requirements. This pilot has provided valuable insight into the current state of genomics in clinical microbiology and public health laboratories across Australia. It will provide a baseline guide for the standardisation of WGS and enable the development of a PT program that allows an ongoing performance benchmark for accreditation of WGS-based test processes.

Recommendations for the introduction of metagenomic next-generation sequencing in clinical virology, part II: bioinformatic analysis and reporting

Metagenomic next-generation sequencing (mNGS) is an untargeted technique for determination of microbial DNA/RNA sequences in a variety of sample types from patients with infectious syndromes. mNGS is still in its early stages of broader translation into clinical applications. To further support the development, implementation, optimization and standardization of mNGS procedures for virus diagnostics, the European Society for Clinical Virology (ESCV) Network on Next-Generation Sequencing (ENNGS) has been established. The aim of ENNGS is to bring together professionals involved in mNGS for viral diagnostics to share methodologies and experiences, and to develop application guidelines. Following the ENNGS publication Recommendations for the introduction of mNGS in clinical virology, part I: wet lab procedure in this journal, the current manuscript aims to provide practical recommendations for the bioinformatic analysis of mNGS data and reporting of results to clinicians.

viromeBrowser: A Shiny App for Browsing Virome Sequencing Analysis Results

Experiments in which complex virome sequencing data is generated remain difficult to explore and unpack for scientists without a background in data science. The processing of raw sequencing data by high throughput sequencing workflows usually results in contigs in FASTA format coupled to an annotation file linking the contigs to a reference sequence or taxonomic identifier. The next step is to compare the virome of different samples based on the metadata of the experimental setup and extract sequences of interest that can be used in subsequent analyses. The viromeBrowser is an application written in the opensource R shiny framework that was developed in collaboration with end-users and is focused on three common data analysis steps. First, the application allows interactive filtering of annotations by default or custom quality thresholds. Next, multiple samples can be visualized to facilitate comparison of contig annotations based on sample specific metadata values. Last, the application makes it easy for users to extract sequences of interest in FASTA format. With the interactive features in the viromeBrowser we aim to enable scientists without a data science background to compare and extract annotation data and sequences from virome sequencing analysis results.

Next-generation diagnostics: virus capture facilitates a sensitive viral diagnosis for epizootic and zoonotic pathogens including SARS-CoV-2

BACKGROUND

The detection of pathogens in clinical and environmental samples using high-throughput sequencing (HTS) is often hampered by large amounts of background information, which is especially true for viruses with small genomes. Enormous sequencing depth can be necessary to compile sufficient information for identification of a certain pathogen. Generic HTS combining with in-solution capture enrichment can markedly increase the sensitivity for virus detection in complex diagnostic samples.

METHODS

A virus panel based on the principle of biotinylated RNA baits was developed for specific capture enrichment of epizootic and zoonotic viruses (VirBaits). The VirBaits set was supplemented by a SARS-CoV-2 predesigned bait set for testing recent SARS-CoV-2-positive samples. Libraries generated from complex samples were sequenced via generic HTS (without enrichment) and afterwards enriched with the VirBaits set. For validation, an internal proficiency test for emerging epizootic and zoonotic viruses (African swine fever virus, Ebolavirus, Marburgvirus, Nipah henipavirus, Rift Valley fever virus) was conducted.

RESULTS

The VirBaits set consists of 177,471 RNA baits (80-mer) based on about 18,800 complete viral genomes targeting 35 epizootic and zoonotic viruses. In all tested samples, viruses with both DNA and RNA genomes were clearly enriched ranging from about 10-fold to 10,000-fold for viruses including distantly related viruses with at least 72% overall identity to viruses represented in the bait set. Viruses showing a lower overall identity (38% and 46%) to them were not enriched but could nonetheless be detected based on capturing conserved genome regions. The internal proficiency test supports the improved virus detection using the combination of HTS plus targeted enrichment but also points to the risk of cross-contamination between samples.

CONCLUSIONS

The VirBaits approach showed a high diagnostic performance, also for distantly related viruses. The bait set is modular and expandable according to the favored diagnostics, health sector, or research question. The risk of cross-contamination needs to be taken into consideration. The application of the RNA-baits principle turned out to be user friendly, and even non-experts can easily use the VirBaits workflow. The rapid extension of the established VirBaits set adapted to actual outbreak events is possible as shown for SARS-CoV-2. Video abstract.

Next-generation diagnostics: virus capture facilitates a sensitive viral diagnosis for epizootic and zoonotic pathogens including SARS-CoV-2

BACKGROUND

The detection of pathogens in clinical and environmental samples using high-throughput sequencing (HTS) is often hampered by large amounts of background information, which is especially true for viruses with small genomes. Enormous sequencing depth can be necessary to compile sufficient information for identification of a certain pathogen. Generic HTS combining with in-solution capture enrichment can markedly increase the sensitivity for virus detection in complex diagnostic samples.

METHODS

A virus panel based on the principle of biotinylated RNA baits was developed for specific capture enrichment of epizootic and zoonotic viruses (VirBaits). The VirBaits set was supplemented by a SARS-CoV-2 predesigned bait set for testing recent SARS-CoV-2-positive samples. Libraries generated from complex samples were sequenced via generic HTS (without enrichment) and afterwards enriched with the VirBaits set. For validation, an internal proficiency test for emerging epizootic and zoonotic viruses (African swine fever virus, Ebolavirus, Marburgvirus, Nipah henipavirus, Rift Valley fever virus) was conducted.

RESULTS

The VirBaits set consists of 177,471 RNA baits (80-mer) based on about 18,800 complete viral genomes targeting 35 epizootic and zoonotic viruses. In all tested samples, viruses with both DNA and RNA genomes were clearly enriched ranging from about 10-fold to 10,000-fold for viruses including distantly related viruses with at least 72% overall identity to viruses represented in the bait set. Viruses showing a lower overall identity (38% and 46%) to them were not enriched but could nonetheless be detected based on capturing conserved genome regions. The internal proficiency test supports the improved virus detection using the combination of HTS plus targeted enrichment but also points to the risk of cross-contamination between samples.

CONCLUSIONS

The VirBaits approach showed a high diagnostic performance, also for distantly related viruses. The bait set is modular and expandable according to the favored diagnostics, health sector, or research question. The risk of cross-contamination needs to be taken into consideration. The application of the RNA-baits principle turned out to be user friendly, and even non-experts can easily use the VirBaits workflow. The rapid extension of the established VirBaits set adapted to actual outbreak events is possible as shown for SARS-CoV-2. Video abstract.

PACIFIC: a lightweight deep-learning classifier of SARS-CoV-2 and co-infecting RNA viruses

Viral co-infections occur in COVID-19 patients, potentially impacting disease progression and severity. However, there is currently no dedicated method to identify viral co-infections in patient RNA-seq data. We developed PACIFIC, a deep-learning algorithm that accurately detects SARS-CoV-2 and other common RNA respiratory viruses from RNA-seq data. Using in silico data, PACIFIC recovers the presence and relative concentrations of viruses with > 99% precision and recall. PACIFIC accurately detects SARS-CoV-2 and other viral infections in 63 independent in vitro cell culture and patient datasets. PACIFIC is an end-to-end tool that enables the systematic monitoring of viral infections in the current global pandemic.

A theoretical and generalized approach for the assessment of the sample-specific limit of detection for clinical metagenomics

Metagenomics is a powerful tool to identify novel or unexpected pathogens, since it is generic and relatively unbiased. The limit of detection (LOD) is a critical parameter for the routine application of methods in the clinical diagnostic context. Although attempts for the determination of LODs for metagenomics next-generation sequencing (mNGS) have been made previously, these were only applicable for specific target species in defined samples matrices. Therefore, we developed and validated a generalized probability-based model to assess the sample-specific LOD of mNGS experiments (LOD_mNGS). Initial rarefaction analyses with datasets of Borna disease virus 1 human encephalitis cases revealed a stochastic behavior of virus read detection. Based on this, we transformed the Bernoulli formula to predict the minimal necessary dataset size to detect one virus read with a probability of 99%. We validated the formula with 30 datasets from diseased individuals, resulting in an accuracy of 99.1% and an average of 4.5 ± 0.4 viral reads found in the calculated minimal dataset size. We demonstrated by modeling the virus genome size, virus-, and total RNA-concentration that the main determinant of mNGS sensitivity is the virus-sample background ratio. The predicted LOD_mNGS for the respective pathogenic virus in the datasets were congruent with the virus-concentration determined by RT-qPCR. Theoretical assumptions were further confirmed by correlation analysis of mNGS and RT-qPCR data from the samples of the analyzed datasets. This approach should guide standardization of mNGS application, due to the generalized concept of LOD_mNGS.

Metagenomics-Based Proficiency Test of Smoked Salmon Spiked with a Mock Community

An inter-laboratory proficiency test was organized to assess the ability of participants to perform shotgun metagenomic sequencing of cold smoked salmon, experimentally spiked with a mock community composed of six bacteria, one parasite, one yeast, one DNA, and two RNA viruses. Each participant applied its in-house wet-lab workflow(s) to obtain the metagenomic dataset(s), which were then collected and analyzed using MG-RAST. A total of 27 datasets were analyzed. Sample pre-processing, DNA extraction protocol, library preparation kit, and sequencing platform, influenced the abundance of specific microorganisms of the mock community. Our results highlight that despite differences in wet-lab protocols, the reads corresponding to the mock community members spiked in the cold smoked salmon, were both detected and quantified in terms of relative abundance, in the metagenomic datasets, proving the suitability of shotgun metagenomic sequencing as a genomic tool to detect microorganisms belonging to different domains in the same food matrix. The implementation of standardized wet-lab protocols would highly facilitate the comparability of shotgun metagenomic sequencing dataset across laboratories and sectors. Moreover, there is a need for clearly defining a sequencing reads threshold, to consider pathogens as detected or undetected in a food sample.

Perspective on Proteomics for Virus Detection in Clinical Samples

One of the most widely used methods to detect an acute viral infection in clinical specimens is diagnostic real-time polymerase chain reaction. However, because of the COVID-19 pandemic, mass-spectrometry-based proteomics is currently being discussed as a potential diagnostic method for viral infections. Because proteomics is not yet applied in routine virus diagnostics, here we discuss its potential to detect viral infections. Apart from theoretical considerations, the current status and technical limitations are considered. Finally, the challenges that have to be overcome to establish proteomics in routine virus diagnostics are highlighted.

Proficiency Testing of Metagenomics-Based Detection of Food-Borne Pathogens Using a Complex Artificial Sequencing Dataset

Metagenomics-based high-throughput sequencing (HTS) enables comprehensive detection of all species comprised in a sample with a single assay and is becoming a standard method for outbreak investigation. However, unlike real-time PCR or serological assays, HTS datasets generated for pathogen detection do not easily provide yes/no answers. Rather, results of the taxonomic read assignment need to be assessed by trained personnel to gain information thereof. Proficiency tests are important instruments of validation, harmonization, and standardization. Within the European Union funded project COMPARE [COllaborative Management Platform for detection and Analyses of (Re-) emerging and foodborne outbreaks in Europe], we conducted a proficiency test to scrutinize the ability to assess diagnostic metagenomics data. An artificial dataset resembling shotgun sequencing of RNA from a sample of contaminated trout was provided to 12 participants with the request to provide a table with per-read taxonomic assignments at species level and a report with a summary and assessment of their findings, considering different categories like pathogen, background, or contaminations. Analysis of the read assignment tables showed that the software used reliably classified the reads taxonomically overall. However, usage of incomplete reference databases or inappropriate data pre-processing caused difficulties. From the combination of the participants' reports with their read assignments, we conclude that, although most species were detected, a number of important taxa were not or not correctly categorized. This implies that knowledge of and awareness for potentially dangerous species and contaminations need to be improved, hence, capacity building for the interpretation of diagnostic metagenomics datasets is necessary.

Novel opportunities for NGS-based one health surveillance of foodborne viruses

Foodborne viral infections rank among the top 5 causes of disease, with noroviruses and hepatitis A causing the greatest burden globally. Contamination of foods by infected food handlers or through environmental pollution are the main sources of foodborne illness, with a lesser role for consumption of products from infected animals. Viral partial genomic sequencing has been used for more than two decades to track foodborne outbreaks and whole genome or metagenomics next-generation-sequencing (NGS) are new additions to the toolbox of food microbiology laboratories. We discuss developments in the field of targeted and metagenomic NGS, with an emphasis on application in food virology, the challenges and possible solutions towards future routine application.

Sequencing of animal viruses: quality data assurance for NGS bioinformatics

Background

Next generation sequencing (NGS) is becoming widely used among diagnostics and research laboratories, and nowadays it is applied to a variety of disciplines, including veterinary virology. The NGS workflow comprises several steps, namely sample processing, library preparation, sequencing and primary/secondary/tertiary bioinformatics (BI) analyses. The latter is constituted by a complex process extremely difficult to standardize, due to the variety of tools and metrics available. Thus, it is of the utmost importance to assess the comparability of results obtained through different methods and in different laboratories. To achieve this goal, we have organized a proficiency test focused on the bioinformatics components for the generation of complete genome sequences of salmonid rhabdoviruses.

Methods

Three partners, that performed virus sequencing using different commercial library preparation kits and NGS platforms, gathered together and shared with each other 75 raw datasets which were analyzed separately by the participants to produce a consensus sequence according to their own bioinformatics pipeline. Results were then compared to highlight discrepancies, and a subset of inconsistencies were investigated more in detail.

Results

In total, we observed 526 discrepancies, of which 39.5% were located at genome termini, 14.1% at intergenic regions and 46.4% at coding regions. Among these, 10 SNPs and 99 indels caused changes in the protein products. Overall reproducibility was 99.94%. Based on the analysis of a subset of inconsistencies investigated more in-depth, manual curation appeared the most critical step affecting sequence comparability, suggesting that the harmonization of this phase is crucial to obtain comparable results. The analysis of a calibrator sample allowed assessing BI accuracy, being 99.983%.

Conclusions

We demonstrated the applicability and the usefulness of BI proficiency testing to assure the quality of NGS data, and recommend a wider implementation of such exercises to guarantee sequence data uniformity among different virology laboratories.

Viral Metagenomics in the Clinical Realm: Lessons Learned from a Swiss-Wide Ring Trial

Shotgun metagenomics using next generation sequencing (NGS) is a promising technique to analyze both DNA and RNA microbial material from patient samples. Mostly used in a research setting, it is now increasingly being used in the clinical realm as well, notably to support diagnosis of viral infections, thereby calling for quality control and the implementation of ring trials (RT) to benchmark pipelines and ensure comparable results. The Swiss NGS clinical virology community therefore decided to conduct a RT in 2018, in order to benchmark current metagenomic workflows used at Swiss clinical virology laboratories, and thereby contribute to the definition of common best practices. The RT consisted of two parts (increments), in order to disentangle the variability arising from the experimental compared to the bioinformatics parts of the laboratory pipeline. In addition, the RT was also designed to assess the impact of databases compared to bioinformatics algorithms on the final results, by asking participants to perform the bioinformatics analysis with a common database, in addition to using their own in-house database. Five laboratories participated in the RT (seven pipelines were tested). We observed that the algorithms had a stronger impact on the overall performance than the choice of the reference database. Our results also suggest that differences in sample preparation can lead to significant differences in the performance, and that laboratories should aim for at least 5-10 Mio reads per sample and use depth of coverage in addition to other interpretation metrics such as the percent of coverage. Performance was generally lower when increasing the number of viruses per sample. The lessons learned from this pilot study will be useful for the development of larger-scale RTs to serve as regular quality control tests for laboratories performing NGS analyses of viruses in a clinical setting.