Divide and Conquer (DC) BLAST: fast and easy BLAST execution within HPC environments

Profiling the BLAST bioinformatics application for load balancing on high-performance computing clusters

BACKGROUND

The Basic Local Alignment Search Tool (BLAST) is a suite of commonly used algorithms for identifying matches between biological sequences. The user supplies a database file and query file of sequences for BLAST to find identical sequences between the two. The typical millions of database and query sequences make BLAST computationally challenging but also well suited for parallelization on high-performance computing clusters. The efficacy of parallelization depends on the data partitioning, where the optimal data partitioning relies on an accurate performance model. In previous studies, a BLAST job was sped up by 27 times by partitioning the database and query among thousands of processor nodes. However, the optimality of the partitioning method was not studied. Unlike BLAST performance models proposed in the literature that usually have problem size and hardware configuration as the only variables, the execution time of a BLAST job is a function of database size, query size, and hardware capability. In this work, the nucleotide BLAST application BLASTN was profiled using three methods: shell-level profiling with the Unix "time" command, code-level profiling with the built-in "profiler" module, and system-level profiling with the Unix "gprof" program. The runtimes were measured for six node types, using six different database files and 15 query files, on a heterogeneous HPC cluster with 500+ nodes. The empirical measurement data were fitted with quadratic functions to develop performance models that were used to guide the data parallelization for BLASTN jobs.

RESULTS

Profiling results showed that BLASTN contains more than 34,500 different functions, but a single function, RunMTBySplitDB, takes 99.12% of the total runtime. Among its 53 child functions, five core functions were identified to make up 92.12% of the overall BLASTN runtime. Based on the performance models, static load balancing algorithms can be applied to the BLASTN input data to minimize the runtime of the longest job on an HPC cluster. Four test cases being run on homogeneous and heterogeneous clusters were tested. Experiment results showed that the runtime can be reduced by 81% on a homogeneous cluster and by 20% on a heterogeneous cluster by re-distributing the workload.

DISCUSSION

Optimal data partitioning can improve BLASTN's overall runtime 5.4-fold in comparison with dividing the database and query into the same number of fragments. The proposed methodology can be used in the other applications in the BLAST+ suite or any other application as long as source code is available.

The final piece of the Triangle of U: Evolution of the tetraploid Brassica carinata genome

Ethiopian mustard (Brassica carinata) is an ancient crop with remarkable stress resilience and a desirable seed fatty acid profile for biofuel uses. Brassica carinata is one of six Brassica species that share three major genomes from three diploid species (AA, BB, and CC) that spontaneously hybridized in a pairwise manner to form three allotetraploid species (AABB, AACC, and BBCC). Of the genomes of these species, that of B. carinata is the least understood. Here, we report a chromosome scale 1.31-Gbp genome assembly with 156.9-fold sequencing coverage for B. carinata, completing the reference genomes comprising the classic Triangle of U, a classical theory of the evolutionary relationships among these six species. Our assembly provides insights into the hybridization event that led to the current B. carinata genome and the genomic features that gave rise to the superior agronomic traits of B. carinata. Notably, we identified an expansion of transcription factor networks and agronomically important gene families. Completion of the Triangle of U comparative genomics platform has allowed us to examine the dynamics of polyploid evolution and the role of subgenome dominance in the domestication and continuing agronomic improvement of B. carinata and other Brassica species.

FastMLST: A Multi-core Tool for Multilocus Sequence Typing of Draft Genome Assemblies

Multilocus Sequence Typing (MLST) is a precise microbial typing approach at the intra-species level for epidemiologic and evolutionary purposes. It operates by assigning a sequence type (ST) identifier to each specimen, based on a combination of alleles of multiple housekeeping genes included in a defined scheme. The use of MLST has multiplied due to the availability of large numbers of genomic sequences and epidemiologic data in public repositories. However, data processing speed has become problematic due to the massive size of modern datasets. Here, we present FastMLST, a tool that is designed to perform PubMLST searches using BLASTn and a divide-and-conquer approach that processes each genome assembly in parallel. The output offered by FastMLST includes a table with the ST, allelic profile, and clonal complex or clade (when available), detected for a query, as well as a multi-FASTA file or a series of FASTA files with the concatenated or single allele sequences detected, respectively. FastMLST was validated with 91 different species, with a wide range of guanine-cytosine content (%GC), genome sizes, and fragmentation levels, and a speed test was performed on 3 datasets with varying genome sizes. Compared with other tools such as mlst, CGE/MLST, MLSTar, and PubMLST, FastMLST takes advantage of multiple processors to simultaneously type up to 28 000 genomes in less than 10 minutes, reducing processing times by at least 3-fold with 100% concordance to PubMLST, if contaminated genomes are excluded from the analysis. The source code, installation instructions, and documentation of FastMLST are available at https://github.com/EnzoAndree/FastMLST

Rapid mitochondrial genome sequencing based on Oxford Nanopore Sequencing and a proxy for vertebrate species identification

Molecular information is crucial for species identification when facing challenging morphology-based specimen identifications. The use of DNA barcodes partially solves this problem, but in some cases when PCR is not an option (i.e., primers are not available, problems in reaction standardization), amplification-free approaches could be an optimal alternative. Recent advances in DNA sequencing, like the MinION device from Oxford Nanopore Technologies (ONT), allow to obtain genomic data with low laboratory and technical requirements, and at a relatively low cost. In this study, we explore ONT sequencing for molecular species identification from a total DNA sample obtained from a neotropical rodent and we also test the technology for complete mitochondrial genome reconstruction via genome skimming. We were able to obtain "de novo" the complete mitogenome of a specimen from the genus Melanomys (Cricetidae: Sigmodontinae) with average depth coverage of 78X using ONT-only data and by combining multiple assembly routines. Our pipeline for an automated species identification was able to identify the sample using unassembled sequence data (raw) in a reasonable computing time, which was substantially reduced when a priori information related to the organism identity was known. Our findings suggest ONT sequencing as a suitable candidate to solve species identification problems in metazoan nonmodel organisms and generate complete mtDNA datasets.

MorphoCatcher: a multiple-alignment based web tool for target selection and designing taxon-specific primers in the loop-mediated isothermal amplification method

BACKGROUND

Advantages of loop-mediated isothermal amplification in molecular diagnostics allow to consider the method as a promising technology of nucleic acid detection in agriculture and medicine. A bioinformatics tool that provides rapid screening and selection of target nucleotide sequences with subsequent taxon-specific primer design toward polymorphic orthologous genes, not only unique or conserved common regions of genome, would contribute to the development of more specific and sensitive diagnostic assays. However, considering features of the original software for primer selection, also known as the PrimerExplorer (Eiken Chemical Co. LTD, Tokyo, Japan), the taxon-specific primer design using multiple sequence alignments of orthologs or even viral genomes with conservative architecture is still complicated.

FINDINGS

Here, MorphoCatcher is introduced as a fast and simple web plugin for PrimerExplorer with a clear interface. It enables an execution of multiple-alignment based search of taxon-specific mutations, visual screening and selection of target sequences, and easy-to-start specific primer design using the PrimerExplorer software. The combination of MorphoCatcher and PrimerExplorer allows to perform processing of the multiple alignments of orthologs for informative sliding-window plot analysis, which is used to identify the sequence regions with a high density of taxon-specific mutations and cover them by the primer ends for better specificity of amplification.

CONCLUSIONS

We hope that this new bioinformatics tool developed for target selection and taxon-specific primer design, called the MorphoCatcher, will gain more popularity of the loop-mediated isothermal amplification method for molecular diagnostics community. MorphoCatcher is a simple web plugin tool for the PrimerExplorer software which is freely available only for non-commercial and academic users at http://morphocatcher.ru.