Deep Learning-Based Automatic Duckweed Counting Using StarDist and Its Application on Measuring Growth Inhibition Potential of Rare Earth Elements as Contaminants of Emerging Concerns

In recent years, there have been efforts to utilize surface water as a power source, material, and food. However, these efforts are impeded due to the vast amounts of contaminants and emerging contaminants introduced by anthropogenic activities. Herbicides such as Glyphosate and Glufosinate are commonly known to contaminate surface water through agricultural industries. In contrast, some emerging contaminants, such as rare earth elements, have started to enter the surface water from the production and waste of electronic products. Duckweeds are angiosperms from the Lemnaceae family and have been used for toxicity tests in aquatic environments, mainly those from the genus Lemna, and have been approved by OECD. In this study, we used duckweed from the genus Wolffia, which is smaller and considered a good indicator of metal pollutants in the aquatic environment. The growth rate of duckweed is the most common endpoint in observing pollutant toxicity. In order to observe and mark the fronds automatically, we used StarDist, a machine learning-based tool. StarDist is available as a plugin in ImageJ, simplifying and assisting the counting process. Python also helps arrange, manage, and calculate the inhibition percentage after duckweeds are exposed to contaminants. The toxicity test results showed Dysprosium to be the most toxic, with an IC50 value of 14.6 ppm, and Samarium as the least toxic, with an IC50 value of 279.4 ppm. In summary, we can provide a workflow for automatic frond counting using StarDist integrated with ImageJ and Python to simplify the detection, counting, data management, and calculation process.

Strategies for Enhancing the Multi-Stage Classification Performances of HER2 Breast Cancer from Hematoxylin and Eosin Images

Breast cancer is a significant health concern among women. Prompt diagnosis can diminish the mortality rate and direct patients to take steps for cancer treatment. Recently, deep learning has been employed to diagnose breast cancer in the context of digital pathology. To help in this area, a transfer learning-based model called 'HE-HER2Net' has been proposed to diagnose multiple stages of HER2 breast cancer (HER2-0, HER2-1+, HER2-2+, HER2-3+) on H&E (hematoxylin & eosin) images from the BCI dataset. HE-HER2Net is the modified version of the Xception model, which is additionally comprised of global average pooling, several batch normalization layers, dropout layers, and dense layers with a swish activation function. This proposed model exceeds all existing models in terms of accuracy (0.87), precision (0.88), recall (0.86), and AUC score (0.98) immensely. In addition, our proposed model has been explained through a class-discriminative localization technique using Grad-CAM to build trust and to make the model more transparent. Finally, nuclei segmentation has been performed through the StarDist method.

StarDist Image Segmentation Improves Circulating Tumor Cell Detection

Simple Summary

Automated enumeration of circulating tumor cells (CTC) from immunofluorescence images starts with a selection of areas containing potential CTC. The CellSearch system has a built-in selection algorithm that has been observed to fail in samples with high cell density, thereby underestimating the true CTC load. We evaluated the deep learning method StarDist for the selection of possible CTC. In whole blood sample images, StarDist recovered 99.95% of CTC detected by CellSearch and segmented 10% additional CTC. In diagnostic leukapheresis (DLA) samples, StarDist segmented 20% additional CTC and performed well, whereas CellSearch had serious failures in 9% of samples.

Abstract

After a CellSearch-processed circulating tumor cell (CTC) sample is imaged, a segmentation algorithm selects nucleic acid positive (DAPI+), cytokeratin-phycoerythrin expressing (CK-PE+) events for further review by an operator. Failures in this segmentation can result in missed CTCs. The CellSearch segmentation algorithm was not designed to handle samples with high cell density, such as diagnostic leukapheresis (DLA) samples. Here, we evaluate deep-learning-based segmentation method StarDist as an alternative to the CellSearch segmentation. CellSearch image archives from 533 whole blood samples and 601 DLA samples were segmented using CellSearch and StarDist and inspected visually. In 442 blood samples from cancer patients, StarDist segmented 99.95% of CTC segmented by CellSearch, produced good outlines for 98.3% of these CTC, and segmented 10% more CTC than CellSearch. Visual inspection of the segmentations of DLA images showed that StarDist continues to perform well when the cell density is very high, whereas CellSearch failed and generated extremely large segmentations (up to 52% of the sample surface). Moreover, in a detailed examination of seven DLA samples, StarDist segmented 20% more CTC than CellSearch. Segmentation is a critical first step for CTC enumeration in dense samples and StarDist segmentation convincingly outperformed CellSearch segmentation.

A deep learning approach to quantify auditory hair cells

Hearing loss affects millions of people worldwide. Yet, there are still no curative therapies for sensorineural hearing loss. Frequent causes of sensorineural hearing loss are due to damage or loss of the sensory hair cells, the spiral ganglion neurons, or the synapses between them. Culturing the organ of Corti allows the study of all these structures in an experimental model, which is easy to manipulate. Therefore, the in vitro culture of the neonatal mammalian organ of Corti remains a frequently used experimental system, in which hair cell survival is routinely assessed. However, the analysis of the surviving hair cells is commonly performed via manual counting, which is a time-consuming process and the inter-rater reliability can be an issue. Here, we describe a deep learning approach to quantify hair cell survival in the murine organ of Corti explants. We used StarDist, a publicly available platform and plugin for Fiji (Fiji is just ImageJ), to train and apply our own custom deep learning model. We successfully validated our model in untreated, cisplatin, and gentamicin treated organ of Corti explants. Therefore, deep learning is a valuable approach for quantifying hair cell survival in organ of Corti explants. Moreover, we also demonstrate how the publicly available Fiji plugin StarDist can be efficiently used for this purpose.

Application of convolutional neural networks towards nuclei segmentation in localization-based super-resolution fluorescence microscopy images

BACKGROUND

Automated segmentation of nuclei in microscopic images has been conducted to enhance throughput in pathological diagnostics and biological research. Segmentation accuracy and speed has been significantly enhanced with the advent of convolutional neural networks. A barrier in the broad application of neural networks to nuclei segmentation is the necessity to train the network using a set of application specific images and image labels. Previous works have attempted to create broadly trained networks for universal nuclei segmentation; however, such networks do not work on all imaging modalities, and best results are still commonly found when the network is retrained on user specific data. Stochastic optical reconstruction microscopy (STORM) based super-resolution fluorescence microscopy has opened a new avenue to image nuclear architecture at nanoscale resolutions. Due to the large size and discontinuous features typical of super-resolution images, automatic nuclei segmentation can be difficult. In this study, we apply commonly used networks (Mask R-CNN and UNet architectures) towards the task of segmenting super-resolution images of nuclei. First, we assess whether networks broadly trained on conventional fluorescence microscopy datasets can accurately segment super-resolution images. Then, we compare the resultant segmentations with results obtained using networks trained directly on our super-resolution data. We next attempt to optimize and compare segmentation accuracy using three different neural network architectures.

RESULTS

Results indicate that super-resolution images are not broadly compatible with neural networks trained on conventional bright-field or fluorescence microscopy images. When the networks were trained on super-resolution data, however, we attained nuclei segmentation accuracies (F1-Score) in excess of 0.8, comparable to past results found when conducting nuclei segmentation on conventional fluorescence microscopy images. Overall, we achieved the best results utilizing the Mask R-CNN architecture.

CONCLUSIONS

We found that convolutional neural networks are powerful tools capable of accurately and quickly segmenting localization-based super-resolution microscopy images of nuclei. While broadly trained and widely applicable segmentation algorithms are desirable for quick use with minimal input, optimal results are still found when the network is both trained and tested on visually similar images. We provide a set of Colab notebooks to disseminate the software into the broad scientific community ( https://github.com/YangLiuLab/Super-Resolution-Nuclei-Segmentation ).

Automated cell tracking using StarDist and TrackMate

The ability of cells to migrate is a fundamental physiological process involved in embryonic development, tissue homeostasis, immune surveillance, and wound healing. Therefore, the mechanisms governing cellular locomotion have been under intense scrutiny over the last 50 years. One of the main tools of this scrutiny is live-cell quantitative imaging, where researchers image cells over time to study their migration and quantitatively analyze their dynamics by tracking them using the recorded images. Despite the availability of computational tools, manual tracking remains widely used among researchers due to the difficulty setting up robust automated cell tracking and large-scale analysis. Here we provide a detailed analysis pipeline illustrating how the deep learning network StarDist can be combined with the popular tracking software TrackMate to perform 2D automated cell tracking and provide fully quantitative readouts. Our proposed protocol is compatible with both fluorescent and widefield images. It only requires freely available and open-source software (ZeroCostDL4Mic and Fiji), and does not require any coding knowledge from the users, making it a versatile and powerful tool for the field. We demonstrate this pipeline's usability by automatically tracking cancer cells and T cells using fluorescent and brightfield images. Importantly, we provide, as supplementary information, a detailed step-by-step protocol to allow researchers to implement it with their images.

OpSeF: Open Source Python Framework for Collaborative Instance Segmentation of Bioimages

Various pre-trained deep learning models for the segmentation of bioimages have been made available as developer-to-end-user solutions. They are optimized for ease of use and usually require neither knowledge of machine learning nor coding skills. However, individually testing these tools is tedious and success is uncertain. Here, we present the Open Segmentation Framework (OpSeF), a Python framework for deep learning-based instance segmentation. OpSeF aims at facilitating the collaboration of biomedical users with experienced image analysts. It builds on the analysts' knowledge in Python, machine learning, and workflow design to solve complex analysis tasks at any scale in a reproducible, well-documented way. OpSeF defines standard inputs and outputs, thereby facilitating modular workflow design and interoperability with other software. Users play an important role in problem definition, quality control, and manual refinement of results. OpSeF semi-automates preprocessing, convolutional neural network (CNN)-based segmentation in 2D or 3D, and postprocessing. It facilitates benchmarking of multiple models in parallel. OpSeF streamlines the optimization of parameters for pre- and postprocessing such, that an available model may frequently be used without retraining. Even if sufficiently good results are not achievable with this approach, intermediate results can inform the analysts in the selection of the most promising CNN-architecture in which the biomedical user might invest the effort of manually labeling training data. We provide Jupyter notebooks that document sample workflows based on various image collections. Analysts may find these notebooks useful to illustrate common segmentation challenges, as they prepare the advanced user for gradually taking over some of their tasks and completing their projects independently. The notebooks may also be used to explore the analysis options available within OpSeF in an interactive way and to document and share final workflows. Currently, three mechanistically distinct CNN-based segmentation methods, the U-Net implementation used in Cellprofiler 3.0, StarDist, and Cellpose have been integrated within OpSeF. The addition of new networks requires little; the addition of new models requires no coding skills. Thus, OpSeF might soon become both an interactive model repository, in which pre-trained models might be shared, evaluated, and reused with ease.