Deep learning-enabled multi-organ segmentation in whole-body mouse scans

Deep learning-based detection of bacterial swarm motion using a single image

Motility is a fundamental characteristic of bacteria. Distinguishing between swarming and swimming, the two principal forms of bacterial movement, holds significant conceptual and clinical relevance. Conventionally, the detection of bacterial swarming involves inoculating samples on an agar surface and observing colony expansion, which is qualitative, time-intensive, and requires additional testing to rule out other motility forms. A recent methodology that differentiates swarming and swimming motility in bacteria using circular confinement offers a rapid approach to detecting swarming. However, it still heavily depends on the observer's expertise, making the process labor-intensive, costly, slow, and susceptible to inevitable human bias. To address these limitations, we developed a deep learning-based swarming classifier that rapidly and autonomously predicts swarming probability using a single blurry image. Compared with traditional video-based, manually processed approaches, our method is particularly suited for high-throughput environments and provides objective, quantitative assessments of swarming probability. The swarming classifier demonstrated in our work was trained on Enterobacter sp. SM3 and showed good performance when blindly tested on new swarming (positive) and swimming (negative) test images of SM3, achieving a sensitivity of 97.44% and a specificity of 100%. Furthermore, this classifier demonstrated robust external generalization capabilities when applied to unseen bacterial species, such as Serratia marcescens DB10 and Citrobacter koseri H6. This competitive performance indicates the potential to adapt our approach for diagnostic applications through portable devices, which would facilitate rapid, objective, on-site screening for bacterial swarming motility, potentially enhancing the early detection and treatment assessment of various diseases, including inflammatory bowel diseases (IBD) and urinary tract infections (UTI).

Streamlining asymmetry quantification in fetal mouse imaging: A semi-automated pipeline supported by expert guidance

BACKGROUND

Asymmetry is a key feature of numerous developmental disorders and in phenotypic screens is often used as a readout for environmental or genetic perturbations. A better understanding of the genetic basis of asymmetry and its relationship to disease susceptibility will help unravel the complex genetic and environmental factors and their interactions that increase risk in a range of developmental disorders. Large-scale imaging datasets offer opportunities to work with sample sizes necessary to detect and quantify differences in morphology beyond severe deformities but also pose challenges to manual phenotyping protocols.

RESULTS

We introduce a tool for quantifying asymmetry in 3D images and apply it to explore the role of genes contributing to abnormal asymmetry by deep phenotyping 3D fetal microCT images from knockout strains acquired as part of the Knockout Mouse Phenotyping Program. Four knockout strains: Ccdc186, Acvr2a, Nhlh1, and Fam20c were identified with highly significant asymmetry in craniofacial regions, making them good candidates for further analysis.

CONCLUSION

In this work, we demonstrate an open-source, semi-automated tool to quantify the asymmetry of craniofacial structures that integrates expert anatomical knowledge. This tool can detect abnormally asymmetric phenotypes in fetal mice to explore the relationship between facial asymmetry, perturbed development, and developmental instability.

Enhancing CT image segmentation accuracy through ensemble loss function optimization

BACKGROUND

In CT-based medical image segmentation, the choice of loss function profoundly impacts the training efficacy of deep neural networks. Traditional loss functions like cross entropy (CE), Dice, Boundary, and TopK each have unique strengths and limitations, often introducing biases when used individually.

PURPOSE

This study aims to enhance segmentation accuracy by optimizing ensemble loss functions, thereby addressing the biases and limitations of single loss functions and their linear combinations.

METHODS

We implemented a comprehensive evaluation of loss function combinations by integrating CE, Dice, Boundary, and TopK loss functions through both loss-level linear combination and model-level ensemble methods. Our approach utilized two state-of-the-art 3D segmentation architectures, Attention U-Net (AttUNet) and SwinUNETR, to test the impact of these methods. The study was conducted on two large CT dataset cohorts: an institutional dataset containing pelvic organ segmentations, and a public dataset consisting of multiple organ segmentations. All the models were trained from scratch with different loss settings, and performance was evaluated using Dice similarity coefficient (DSC), Hausdorff distance (HD), and average surface distance (ASD). In the ensemble approach, both static averaging and learnable dynamic weighting strategies were employed to combine the outputs of models trained with different loss functions.

RESULTS

Extensive experiments revealed the following: (1) the linear combination of loss functions achieved results comparable to those of single loss-driven methods; (2) compared to the best non-ensemble methods, ensemble-based approaches resulted in a 2%-7% increase in DSC scores, along with notable reductions in HD (e.g., a 19.1% reduction for rectum segmentation using SwinUNETR) and ASD (e.g., a 49.0% reduction for prostate segmentation using AttUNet); (3) the learnable ensemble approach with optimized weights produced finer details in predicted masks, as confirmed by qualitative analyses; and (4) the learnable ensemble consistently outperforms the static ensemble across most metrics (DSC, HD, ASD) for both AttUNet and SwinUNETR architectures.

CONCLUSIONS

Our findings support the efficacy of using ensemble models with optimized weights to improve segmentation accuracy, highlighting the potential for broader applications in automated medical image analysis.

Development of anatomically accurate digital organ models for surgical simulation and training

Advancements in robotics and other technological innovations have accelerated the development of surgical procedures, increasing the demand for training environments that accurately replicate human anatomy. This study developed a system that utilizes the AutoSegmentator extension of 3D Slicer, based on nnU-Net, a state-of-the-art deep learning framework for automatic organ extraction, to import automatically extracted organ surface data into CAD software along with original DICOM-derived images. This system allows medical experts to manually refine the automatically extracted data, making it more accurate and closer to the ideal dataset. First, Python programming is used to automatically generate and save JPEG-format image data from DICOM data for display in Blender. Next, DICOM data imported into 3D Slicer is processed by AutoSegmentator to extract surface data of 104 organs in bulk, which is then exported in STL format. In Blender, a custom-developed Python script aligns the image data and organ surface data within the same 3D space, ensuring accurate spatial coordinates. By using Blender's CAD functionality within this space, the automatically extracted organ boundaries can be manually adjusted based on the image data, resulting in more precise organ surface data. Additionally, organs and blood vessels that cannot be automatically extracted can be newly created and added by referencing the image data. Through this process, a comprehensive anatomical dataset encompassing all required organs and blood vessels can be constructed. The dataset created with this system is easily customizable and can be applied to various surgical simulations, including 3D-printed simulators, hybrid simulators that incorporate animal organs, and surgical simulators utilizing augmented reality (AR). Furthermore, this system is built entirely using open-source, free software, providing high reproducibility, flexibility, and accessibility. By using this system, medical professionals can actively participate in the design and data processing of surgical simulation systems, leading to shorter development times and reduced costs.

Unveiling Another Dimension: Advanced Visualization of Cancer Invasion and Metastasis via Micro-CT Imaging

Invasion and metastasis are well-known hallmarks of cancer, with metastatic disease accounting for 60% to 90% of cancer-related deaths [...].

A robust deep learning approach for segmenting cortical and trabecular bone from 3D high resolution µCT scans of mouse bone

Recent advancements in deep learning have significantly enhanced the segmentation of high-resolution microcomputed tomography (µCT) bone scans. In this paper, we present the dual-branch attention-based hybrid network (DBAHNet), a deep learning architecture designed for automatically segmenting the cortical and trabecular compartments in 3D µCT scans of mouse tibiae. DBAHNet's hierarchical structure combines transformers and convolutional neural networks to capture long-range dependencies and local features for improved contextual representation. We trained DBAHNet on a limited dataset of 3D µCT scans of mouse tibiae and evaluated its performance on a diverse dataset collected from seven different research studies. This evaluation covered variations in resolutions, ages, mouse strains, drug treatments, surgical procedures, and mechanical loading. DBAHNet demonstrated excellent performance, achieving high accuracy, particularly in challenging scenarios with significantly altered bone morphology. The model's robustness and generalization capabilities were rigorously tested under diverse and unseen conditions, confirming its effectiveness in the automated segmentation of high-resolution µCT mouse tibia scans. Our findings highlight DBAHNet's potential to provide reliable and accurate 3D µCT mouse tibia segmentation, thereby enhancing and accelerating preclinical bone studies in drug development. The model and code are available at https://github.com/bigfahma/DBAHNet .

Prediction of treatment response and outcome of transarterial chemoembolization in patients with hepatocellular carcinoma using artificial intelligence: A systematic review of efficacy

PURPOSE

To perform a systematic literature review of the efficacy of different AI models to predict HCC treatment response to transarterial chemoembolization (TACE), including overall survival (OS) and time to progression (TTP).

METHODS

This systematic review was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines until May 2, 2024.

RESULTS

The systematic review included 23 studies with 4,486 HCC patients. The AI algorithm receiver operator characteristic (ROC) area under the curve (AUC) for predicting HCC response to TACE based on mRECIST criteria ranged from 0.55 to 0.97. Radiomics-models outperformed non-radiomics models (AUCs: 0.79, 95 %CI: 0.75-0.82 vs. 0.73, 0.61-0.77, respectively). The best ML methods used for the prediction of TACE response for HCC patients were CNN, GB, SVM, and RF with AUCs of 0.88 (0.79-0.97), 0.82 (0.71-0.89), 0.8 (0.60-0.87) and 0.8 (0.55-0.96), respectively. Of all predictive feature models, those combining clinic-radiologic features (ALBI grade, BCLC stage, AFP level, tumor diameter, distribution, and peritumoral arterial enhancement) had higher AUCs compared with models based on clinical characteristics alone (0.79, 0.73-0.89; p = 0.04 for CT + clinical, 0.81, 0.75-0.88; p = 0.017 for MRI + clinical versus 0.6, 0.55-0.75 in clinical characteristics alone).

CONCLUSION

Integrating clinic-radiologic features enhances AI models' predictive performance for HCC patient response to TACE, with CNN, GB, SVM, and RF methods outperforming others. Key predictive clinic-radiologic features include ALBI grade, BCLC stage, AFP level, tumor diameter, distribution, and peritumoral arterial enhancement. Multi-institutional studies are needed to improve AI model accuracy, address heterogeneity, and resolve validation issues.

Robust Automated Mouse Micro-CT Segmentation Using Swin UNEt TRansformers

Image-guided mouse irradiation is essential to understand interventions involving radiation prior to human studies. Our objective is to employ Swin UNEt TRansformers (Swin UNETR) to segment native micro-CT and contrast-enhanced micro-CT scans and benchmark the results against 3D no-new-Net (nnU-Net). Swin UNETR reformulates mouse organ segmentation as a sequence-to-sequence prediction task using a hierarchical Swin Transformer encoder to extract features at five resolution levels, and it connects to a Fully Convolutional Neural Network (FCNN)-based decoder via skip connections. The models were trained and evaluated on open datasets, with data separation based on individual mice. Further evaluation on an external mouse dataset acquired on a different micro-CT with lower kVp and higher imaging noise was also employed to assess model robustness and generalizability. The results indicate that Swin UNETR consistently outperforms nnU-Net and AIMOS in terms of the average dice similarity coefficient (DSC) and the Hausdorff distance (HD95p), except in two mice for intestine contouring. This superior performance is especially evident in the external dataset, confirming the model's robustness to variations in imaging conditions, including noise and quality, and thereby positioning Swin UNETR as a highly generalizable and efficient tool for automated contouring in pre-clinical workflows.

How scanning probe microscopy can be supported by artificial intelligence and quantum computing?

The impact of Artificial Intelligence (AI) is rapidly expanding, revolutionizing both science and society. It is applied to practically all areas of life, science, and technology, including materials science, which continuously requires novel tools for effective materials characterization. One of the widely used techniques is scanning probe microscopy (SPM). SPM has fundamentally changed materials engineering, biology, and chemistry by providing tools for atomic-precision surface mapping. Despite its many advantages, it also has some drawbacks, such as long scanning times or the possibility of damaging soft-surface materials. In this paper, we focus on the potential for supporting SPM-based measurements, with an emphasis on the application of AI-based algorithms, especially Machine Learning-based algorithms, as well as quantum computing (QC). It has been found that AI can be helpful in automating experimental processes in routine operations, algorithmically searching for optimal sample regions, and elucidating structure-property relationships. Thus, it contributes to increasing the efficiency and accuracy of optical nanoscopy scanning probes. Moreover, the combination of AI-based algorithms and QC may have enormous potential to enhance the practical application of SPM. The limitations of the AI-QC-based approach were also discussed. Finally, we outline a research path for improving AI-QC-powered SPM. RESEARCH HIGHLIGHTS: Artificial intelligence and quantum computing as support for scanning probe microscopy. The analysis indicates a research gap in the field of scanning probe microscopy. The research aims to shed light into ai-qc-powered scanning probe microscopy.

Development of a PTV margin for preclinical irradiation of orthotopic pancreatic tumors derived from a well-known recipe for humans

In human radiotherapy a safety margin (PTV margin) is essential for successful irradiation and is usually part of clinical treatment planning. In preclinical radiotherapy research with small animals, most uncertainties and inaccuracies are present as well, but according to the literature a margin is used only scarcely. In addition, there is only little experience about the appropriate size of the margin, which should carefully be investigated and considered, since sparing of organs at risk or normal tissue is affected. Here we estimate the needed margin for preclinical irradiation by adapting a well-known human margin recipe from van Herck et al. to the dimensions and requirements of the specimen on a small animal radiation research platform (SARRP). We adjusted the factors of the described formula to the specific challenges in an orthotopic pancreatic tumor mouse model to establish an appropriate margin concept. The SARRP was used with its image-guidance irradiation possibility for arc irradiation with a field size of 10 × 10 mm2 for 5 fractions. Our goal was to irradiate the clinical target volume (CTV) of at least 90% of our mice with at least 95% of the prescribed dose. By carefully analyzing all relevant factors we gain a CTV to planning target volume (PTV) margin of 1.5 mm for our preclinical setup. The stated safety margin is strongly dependent on the exact setting of the experiment and has to be adjusted for other experimental settings. The few stated values in literature correspond well to our result. Even if using margins in the preclinical setting might be an additional challenge, we think it is crucial to use them to produce reliable results and improve the efficacy of radiotherapy.

Universal and extensible language-vision models for organ segmentation and tumor detection from abdominal computed tomography

The advancement of artificial intelligence (AI) for organ segmentation and tumor detection is propelled by the growing availability of computed tomography (CT) datasets with detailed, per-voxel annotations. However, these AI models often struggle with flexibility for partially annotated datasets and extensibility for new classes due to limitations in the one-hot encoding, architectural design, and learning scheme. To overcome these limitations, we propose a universal, extensible framework enabling a single model, termed Universal Model, to deal with multiple public datasets and adapt to new classes (e.g., organs/tumors). Firstly, we introduce a novel language-driven parameter generator that leverages language embeddings from large language models, enriching semantic encoding compared with one-hot encoding. Secondly, the conventional output layers are replaced with lightweight, class-specific heads, allowing Universal Model to simultaneously segment 25 organs and six types of tumors and ease the addition of new classes. We train our Universal Model on 3410 CT volumes assembled from 14 publicly available datasets and then test it on 6173 CT volumes from four external datasets. Universal Model achieves first place on six CT tasks in the Medical Segmentation Decathlon (MSD) public leaderboard and leading performance on the Beyond The Cranial Vault (BTCV) dataset. In summary, Universal Model exhibits remarkable computational efficiency (6× faster than other dataset-specific models), demonstrates strong generalization across different hospitals, transfers well to numerous downstream tasks, and more importantly, facilitates the extensibility to new classes while alleviating the catastrophic forgetting of previously learned classes. Codes, models, and datasets are available at https://github.com/ljwztc/CLIP-Driven-Universal-Model.

Multiscale and multimodal evaluation of autosomal dominant polycystic kidney disease development

Autosomal Dominant Polycystic Kidney Disease (ADPKD) is the most prevalent kidney genetic disorder, producing structural abnormalities and impaired function. This research investigates its evolution on mouse models, utilizing a combination of histology imaging, Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) to evaluate its progression thoroughly. ADPKD has been induced in mice via PKD2 gene knockout, followed by image acquisition at different stages. Histology data provides two-dimensional details, like the cystic area ratio, whereas CT and MRI facilitate three-dimensional temporal monitoring. Our approach allows to quantify the affected tissue at different disease stages through multiple quantitative metrics. A pivotal point is shown at approximately ten weeks after induction, marked by a swift acceleration in disease advancement, and leading to a notable increase in cyst formation. This multimodal strategy augments our comprehension of ADPKD dynamics and suggests the possibility of employing higher-resolution imaging in the future for more accurate volumetric analyses.

Applications of Lightsheet Fluorescence Microscopy by High Numerical Aperture Detection Lens

This Review explores the evolution, improvements, and recent applications of Light Sheet Fluorescence Microscopy (LSFM) in biological research using a high numerical aperture detection objective (lens) for imaging subcellular structures. The Review begins with an overview of the development of LSFM, tracing its evolution from its inception to its current state and emphasizing key milestones and technological advancements over the years. Subsequently, we will discuss various improvements of LSFM techniques, covering advancements in hardware such as illumination strategies, optical designs, and sample preparation methods that have enhanced imaging capabilities and resolution. The advancements in data acquisition and processing are also included, which provides a brief overview of the recent development of artificial intelligence. Fluorescence probes that were commonly used in LSFM will be highlighted, together with some insights regarding the selection of potential probe candidates for future LSFM development. Furthermore, we also discuss recent advances in the application of LSFM with a focus on high numerical aperture detection objectives for various biological studies. For sample preparation techniques, there are discussions regarding fluorescence probe selection, tissue clearing protocols, and some insights into expansion microscopy. Integrated setups such as adaptive optics, single objective modification, and microfluidics will also be some of the key discussion points in this Review. We hope that this comprehensive Review will provide a holistic perspective on the historical development, technical enhancements, and cutting-edge applications of LSFM, showcasing its pivotal role and future potential in advancing biological research.

Self-adaptive deep learning-based segmentation for universal and functional clinical and preclinical CT image analysis

BACKGROUND

Methods to monitor cardiac functioning non-invasively can accelerate preclinical and clinical research into novel treatment options for heart failure. However, manual image analysis of cardiac substructures is resource-intensive and error-prone. While automated methods exist for clinical CT images, translating these to preclinical μCT data is challenging. We employed deep learning to automate the extraction of quantitative data from both CT and μCT images.

METHODS

We collected a public dataset of cardiac CT images of human patients, as well as acquired μCT images of wild-type and accelerated aging mice. The left ventricle, myocardium, and right ventricle were manually segmented in the μCT training set. After template-based heart detection, two separate segmentation neural networks were trained using the nnU-Net framework.

RESULTS

The mean Dice score of the CT segmentation results (0.925 ± 0.019, n = 40) was superior to those achieved by state-of-the-art algorithms. Automated and manual segmentations of the μCT training set were nearly identical. The estimated median Dice score (0.940) of the test set results was comparable to existing methods. The automated volume metrics were similar to manual expert observations. In aging mice, ejection fractions had significantly decreased, and myocardial volume increased by age 24 weeks.

CONCLUSIONS

With further optimization, automated data extraction expands the application of (μ)CT imaging, while reducing subjectivity and workload. The proposed method efficiently measures the left and right ventricular ejection fraction and myocardial mass. With uniform translation between image types, cardiac functioning in diastolic and systolic phases can be monitored in both animals and humans.

Optimizing SUV Analysis: A Multicenter Study on Preclinical FDG-PET/CT Highlights the Impact of Standardization

PURPOSE

Preclinical imaging, with translational potential, lacks a standardized method for defining volumes of interest (VOIs), impacting data reproducibility. The aim of this study was to determine the interobserver variability of VOI sizes and standard uptake values (SUVmean and SUVmax) of different organs using the same [18F]FDG-PET and PET/CT datasets analyzed by multiple observers. In addition, the effect of a standardized analysis approach was evaluated.

PROCEDURES

In total, 12 observers (4 beginners and 8 experts) analyzed identical preclinical [18F]FDG-PET-only and PET/CT datasets according to their local default image analysis protocols for multiple organs. Furthermore, a standardized protocol was defined, including detailed information on the respective VOI size and position for multiple organs, and all observers reanalyzed the PET/CT datasets following this protocol.

RESULTS

Without standardization, significant differences in the SUVmean and SUVmax were found among the observers. Coregistering CT images with PET images improved the comparability to a limited extent. The introduction of a standardized protocol that details the VOI size and position for multiple organs reduced interobserver variability and enhanced comparability.

CONCLUSIONS

The protocol offered clear guidelines and was particularly beneficial for beginners, resulting in improved comparability of SUVmean and SUVmax values for various organs. The study suggested that incorporating an additional VOI template could further enhance the comparability of the findings in preclinical imaging analyses.

Deep 3D histology powered by tissue clearing, omics and AI

To comprehensively understand tissue and organism physiology and pathophysiology, it is essential to create complete three-dimensional (3D) cellular maps. These maps require structural data, such as the 3D configuration and positioning of tissues and cells, and molecular data on the constitution of each cell, spanning from the DNA sequence to protein expression. While single-cell transcriptomics is illuminating the cellular and molecular diversity across species and tissues, the 3D spatial context of these molecular data is often overlooked. Here, I discuss emerging 3D tissue histology techniques that add the missing third spatial dimension to biomedical research. Through innovations in tissue-clearing chemistry, labeling and volumetric imaging that enhance 3D reconstructions and their synergy with molecular techniques, these technologies will provide detailed blueprints of entire organs or organisms at the cellular level. Machine learning, especially deep learning, will be essential for extracting meaningful insights from the vast data. Further development of integrated structural, molecular and computational methods will unlock the full potential of next-generation 3D histology.

Exploring Automated Contouring Across Institutional Boundaries: A Deep Learning Approach with Mouse Micro-CT Datasets

Image-guided mouse irradiation is essential to understand interventions involving radiation prior to human studies. Our objective is to employ Swin UNEt Transformers (Swin UNETR) to segment native micro-CT and contrast-enhanced micro-CT scans and benchmark the results against 3D no-new-Net (nnU-Net). Swin UNETR reformulates mouse organ segmentation as a sequence-to-sequence prediction task, using a hierarchical Swin Transformer encoder to extract features at 5 resolution levels, and connects to a Fully Convolutional Neural Network (FCNN)-based decoder via skip connections. The models were trained and evaluated on open datasets, with data separation based on individual mice. Further evaluation on an external mouse dataset acquired on a different micro-CT with lower kVp and higher imaging noise was also employed to assess model robustness and generalizability. Results indicate that Swin UNETR consistently outperforms nnU-Net and AIMOS in terms of average dice similarity coefficient (DSC) and Hausdorff distance (HD95p), except in two mice of intestine contouring. This superior performance is especially evident in the external dataset, confirming the model's robustness to variations in imaging conditions, including noise and quality, thereby positioning Swin UNETR as a highly generalizable and efficient tool for automated contouring in pre-clinical workflows.

COSST: Multi-Organ Segmentation With Partially Labeled Datasets Using Comprehensive Supervisions and Self-Training

Deep learning models have demonstrated remarkable success in multi-organ segmentation but typically require large-scale datasets with all organs of interest annotated. However, medical image datasets are often low in sample size and only partially labeled, i.e., only a subset of organs are annotated. Therefore, it is crucial to investigate how to learn a unified model on the available partially labeled datasets to leverage their synergistic potential. In this paper, we systematically investigate the partial-label segmentation problem with theoretical and empirical analyses on the prior techniques. We revisit the problem from a perspective of partial label supervision signals and identify two signals derived from ground truth and one from pseudo labels. We propose a novel two-stage framework termed COSST, which effectively and efficiently integrates comprehensive supervision signals with self-training. Concretely, we first train an initial unified model using two ground truth-based signals and then iteratively incorporate the pseudo label signal to the initial model using self-training. To mitigate performance degradation caused by unreliable pseudo labels, we assess the reliability of pseudo labels via outlier detection in latent space and exclude the most unreliable pseudo labels from each self-training iteration. Extensive experiments are conducted on one public and three private partial-label segmentation tasks over 12 CT datasets. Experimental results show that our proposed COSST achieves significant improvement over the baseline method, i.e., individual networks trained on each partially labeled dataset. Compared to the state-of-the-art partial-label segmentation methods, COSST demonstrates consistent superior performance on various segmentation tasks and with different training data sizes.

Social complexity affects cognitive abilities but not brain structure in a Poeciliid fish

Some cognitive abilities are suggested to be the result of a complex social life, allowing individuals to achieve higher fitness through advanced strategies. However, most evidence is correlative. Here, we provide an experimental investigation of how group size and composition affect brain and cognitive development in the guppy (Poecilia reticulata). For 6 months, we reared sexually mature females in one of 3 social treatments: a small conspecific group of 3 guppies, a large heterospecific group of 3 guppies and 3 splash tetras (Copella arnoldi)-a species that co-occurs with the guppy in the wild, and a large conspecific group of 6 guppies. We then tested the guppies' performance in self-control (inhibitory control), operant conditioning (associative learning), and cognitive flexibility (reversal learning) tasks. Using X-ray imaging, we measured their brain size and major brain regions. Larger groups of 6 individuals, both conspecific and heterospecific groups, showed better cognitive flexibility than smaller groups but no difference in self-control and operant conditioning tests. Interestingly, while social manipulation had no significant effect on brain morphology, relatively larger telencephalons were associated with better cognitive flexibility. This suggests alternative mechanisms beyond brain region size enabled greater cognitive flexibility in individuals from larger groups. Although there is no clear evidence for the impact on brain morphology, our research shows that living in larger social groups can enhance cognitive flexibility. This indicates that the social environment plays a role in the cognitive development of guppies.

Deep learning for automatic organ and tumor segmentation in nanomedicine pharmacokinetics

Rationale: Multimodal imaging provides important pharmacokinetic and dosimetry information during nanomedicine development and optimization. However, accurate quantitation is time-consuming, resource intensive, and requires anatomical expertise. Methods: We present NanoMASK: a 3D U-Net adapted deep learning tool capable of rapid, automatic organ segmentation of multimodal imaging data that can output key clinical dosimetry metrics without manual intervention. This model was trained on 355 manually-contoured PET/CT data volumes of mice injected with a variety of nanomaterials and imaged over 48 hours. Results: NanoMASK produced 3-dimensional contours of the heart, lungs, liver, spleen, kidneys, and tumor with high volumetric accuracy (pan-organ average %DSC of 92.5). Pharmacokinetic metrics including %ID/cc, %ID, and SUVmax achieved correlation coefficients exceeding R = 0.987 and relative mean errors below 0.2%. NanoMASK was applied to novel datasets of lipid nanoparticles and antibody-drug conjugates with a minimal drop in accuracy, illustrating its generalizability to different classes of nanomedicines. Furthermore, 20 additional auto-segmentation models were developed using training data subsets based on image modality, experimental imaging timepoint, and tumor status. These were used to explore the fundamental biases and dependencies of auto-segmentation models built on a 3D U-Net architecture, revealing significant differential impacts on organ segmentation accuracy. Conclusions: NanoMASK is an easy-to-use, adaptable tool for improving accuracy and throughput in imaging-based pharmacokinetic studies of nanomedicine. It has been made publicly available to all readers for automatic segmentation and pharmacokinetic analysis across a diverse array of nanoparticles, expediting agent development.

Advances in medical image analysis with vision Transformers: A comprehensive review

The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.

Learning From Partially Labeled Data for Multi-Organ and Tumor Segmentation

Medical image benchmarks for the segmentation of organs and tumors suffer from the partially labeling issue due to its intensive cost of labor and expertise. Current mainstream approaches follow the practice of one network solving one task. With this pipeline, not only the performance is limited by the typically small dataset of a single task, but also the computation cost linearly increases with the number of tasks. To address this, we propose a Transformer based dynamic on-demand network (TransDoDNet) that learns to segment organs and tumors on multiple partially labeled datasets. Specifically, TransDoDNet has a hybrid backbone that is composed of the convolutional neural network and Transformer. A dynamic head enables the network to accomplish multiple segmentation tasks flexibly. Unlike existing approaches that fix kernels after training, the kernels in the dynamic head are generated adaptively by the Transformer, which employs the self-attention mechanism to model long-range organ-wise dependencies and decodes the organ embedding that can represent each organ. We create a large-scale partially labeled Multi-Organ and Tumor Segmentation benchmark, termed MOTS, and demonstrate the superior performance of our TransDoDNet over other competitors on seven organ and tumor segmentation tasks. This study also provides a general 3D medical image segmentation model, which has been pre-trained on the large-scale MOTS benchmark and has demonstrated advanced performance over current predominant self-supervised learning methods.

Technical note: Multi-MATE, a high-throughput platform for automated image-guided small-animal irradiation

BACKGROUND

Small animal irradiation is essential to study the radiation response of new interventions before or parallel to human therapy. Image-guided radiotherapy (IGRT) and intensity-modulated radiotherapy (IMRT) are recently adopted in small animal irradiation to more closely mimic human treatments. However, sophisticated techniques require exceedingly high time, resources, and expertize that are often impractical.

PURPOSE

We propose a high throughput and high precision platform named Multiple Mouse Automated Treatment Environment (Multi-MATE) to streamline image-guided small animal irradiation.

METHODS

Multi-MATE consists of six parallel and hexagonally arranged channels, each equipped with a transfer railing, a 3D-printed immobilization pod, and an electromagnetic control unit, computer-controlled via an Arduino interface. The mouse immobilization pods are transferred along the railings between the home position outside the radiation field and the imaging/irradiation position at the irradiator isocenter. All six immobilization pods are transferred to the isocenter in the proposed workflow for parallel CBCT scans and treatment planning. The immobilization pods are then sequentially transported to the imaging/therapy position for dose delivery. The positioning reproducibility of Multi-MATE are evaluated using CBCT and radiochromic films.

RESULTS

While parallelizing and automating the image-guided small animal radiation delivery, Multi-MATE achieved the average pod position reproducibility of 0.17 ± 0.04 mm in the superior-inferior direction, 0.20 ± 0.04 mm in the left-right direction, and 0.12 ± 0.02mm in the anterior-posterior direction in repeated CBCT tests. Additionally, in image-guided dose delivery tasks, Multi-MATE demonstrated the positioning reproducibility of 0.17 ± 0.06 mm in the superior-inferior direction, 0.19 ± 0.06 mm in the left-right direction.

CONCLUSIONS

We designed, fabricated, and tested a novel automated irradiation platform, Multi-MATE to accelerate and automate image-guided small animal irradiation. The automated platform minimizes human operation and achieves high setup reproducibility and image-guided dose delivery accuracy. Multi-MATE thus removes a major barrier to implementing high-precision preclinical radiation research.

Multi-center study on predicting breast cancer lymph node status from core needle biopsy specimens using multi-modal and multi-instance deep learning

The objective of our study is to develop a deep learning model based on clinicopathological data and digital pathological image of core needle biopsy specimens for predicting breast cancer lymph node metastasis. We collected 3701 patients from the Fourth Hospital of Hebei Medical University and 190 patients from four medical centers in Hebei Province. Integrating clinicopathological data and image features build multi-modal and multi-instance (MMMI) deep learning model to obtain the final prediction. For predicting with or without lymph node metastasis, the AUC was 0.770, 0.709, 0.809 based on the clinicopathological features, WSI and MMMI, respectively. For predicting four classification of lymph node status (no metastasis, isolated tumor cells (ITCs), micrometastasis, and macrometastasis), the prediction based on clinicopathological features, WSI and MMMI were compared. The AUC for no metastasis was 0.770, 0.709, 0.809, respectively; ITCs were 0.619, 0.531, 0.634, respectively; micrometastasis were 0.636, 0.617, 0.691, respectively; and macrometastasis were 0.748, 0.691, 0.758, respectively. The MMMI model achieved the highest prediction accuracy. For prediction of different molecular types of breast cancer, MMMI demonstrated a better prediction accuracy for any type of lymph node status, especially in the molecular type of triple negative breast cancer (TNBC). In the external validation sets, MMMI also showed better prediction accuracy in the four classification, with AUC of 0.725, 0.757, 0.525, and 0.708, respectively. Finally, we developed a breast cancer lymph node metastasis prediction model based on a MMMI model. Through all cases tests, the results showed that the overall prediction ability was high.

Breast-Conserving Surgery Margin Guidance Using Micro-Computed Tomography: Challenges When Imaging Radiodense Resection Specimens

BACKGROUND

Breast-conserving surgery (BCS) is an integral component of early-stage breast cancer treatment, but costly reexcision procedures are common due to the high prevalence of cancer-positive margins on primary resections. A need exists to develop and evaluate improved methods of margin assessment to detect positive margins intraoperatively.

METHODS

A prospective trial was conducted through which micro-computed tomography (micro-CT) with radiological interpretation by three independent readers was evaluated for BCS margin assessment. Results were compared to standard-of-care intraoperative margin assessment (i.e., specimen palpation and radiography [abbreviated SIA]) for detecting cancer-positive margins.

RESULTS

Six hundred margins from 100 patients were analyzed. Twenty-one margins in 14 patients were pathologically positive. On analysis at the specimen-level, SIA yielded a sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of 42.9%, 76.7%, 23.1%, and 89.2%, respectively. SIA correctly identified six of 14 margin-positive cases with a 23.5% false positive rate (FPR). Micro-CT readers achieved sensitivity, specificity, PPV, and NPV ranges of 35.7-50.0%, 55.8-68.6%, 15.6-15.8%, and 86.8-87.3%, respectively. Micro-CT readers correctly identified five to seven of 14 margin-positive cases with an FPR range of 31.4-44.2%. If micro-CT scanning had been combined with SIA, up to three additional margin-positive specimens would have been identified.

DISCUSSION

Micro-CT identified a similar proportion of margin-positive cases as standard specimen palpation and radiography, but due to difficulty distinguishing between radiodense fibroglandular tissue and cancer, resulted in a higher proportion of false positive margin assessments.

Shortwave infrared fluorescence imaging of peripheral organs in awake and freely moving mice

Extracting biological information from awake and unrestrained mice is imperative to in vivo basic and pre-clinical research. Accordingly, imaging methods which preclude invasiveness, anesthesia, and/or physical restraint enable more physiologically relevant biological data extraction by eliminating these extrinsic confounders. In this article, we discuss the recent development of shortwave infrared (SWIR) fluorescent imaging to visualize peripheral organs in freely-behaving mice, as well as propose potential applications of this imaging modality in the neurosciences.

Shortwave infrared (SWIR) fluorescence imaging of peripheral organs in awake and freely moving mice

Extracting biological information from awake and unrestrained mice is imperative to in vivo basic and pre-clinical research. Accordingly, imaging methods which preclude invasiveness, anesthesia, and/or physical restraint enable more physiologically relevant biological data extraction by eliminating these extrinsic confounders. In this article we discuss the recent development of shortwave infrared (SWIR) fluorescent imaging to visualize peripheral organs in freely-behaving mice, as well as propose potential applications of this imaging modality in the neurosciences.

Roadmap for precision preclinical x-ray radiation studies

This Roadmap paper covers the field of precision preclinical x-ray radiation studies in animal models. It is mostly focused on models for cancer and normal tissue response to radiation, but also discusses other disease models. The recent technological evolution in imaging, irradiation, dosimetry and monitoring that have empowered these kinds of studies is discussed, and many developments in the near future are outlined. Finally, clinical translation and reverse translation are discussed.

Improved Repeatability of Mouse Tibia Volume Segmentation in Murine Myelofibrosis Model Using Deep Learning

A murine model of myelofibrosis in tibia was used in a co-clinical trial to evaluate segmentation methods for application of image-based biomarkers to assess disease status. The dataset (32 mice with 157 3D MRI scans including 49 test-retest pairs scanned on consecutive days) was split into approximately 70% training, 10% validation, and 20% test subsets. Two expert annotators (EA1 and EA2) performed manual segmentations of the mouse tibia (EA1: all data; EA2: test and validation). Attention U-net (A-U-net) model performance was assessed for accuracy with respect to EA1 reference using the average Jaccard index (AJI), volume intersection ratio (AVI), volume error (AVE), and Hausdorff distance (AHD) for four training scenarios: full training, two half-splits, and a single-mouse subsets. The repeatability of computer versus expert segmentations for tibia volume of test-retest pairs was assessed by within-subject coefficient of variance (%wCV). A-U-net models trained on full and half-split training sets achieved similar average accuracy (with respect to EA1 annotations) for test set: AJI = 83-84%, AVI = 89-90%, AVE = 2-3%, and AHD = 0.5 mm-0.7 mm, exceeding EA2 accuracy: AJ = 81%, AVI = 83%, AVE = 14%, and AHD = 0.3 mm. The A-U-net model repeatability wCV [95% CI]: 3 [2, 5]% was notably better than that of expert annotators EA1: 5 [4, 9]% and EA2: 8 [6, 13]%. The developed deep learning model effectively automates murine bone marrow segmentation with accuracy comparable to human annotators and substantially improved repeatability.

Deep learning enabled multi-organ segmentation of mouse embryos

The International Mouse Phenotyping Consortium (IMPC) has generated a large repository of three-dimensional (3D) imaging data from mouse embryos, providing a rich resource for investigating phenotype/genotype interactions. While the data is freely available, the computing resources and human effort required to segment these images for analysis of individual structures can create a significant hurdle for research. In this paper, we present an open source, deep learning-enabled tool, Mouse Embryo Multi-Organ Segmentation (MEMOS), that estimates a segmentation of 50 anatomical structures with a support for manually reviewing, editing, and analyzing the estimated segmentation in a single application. MEMOS is implemented as an extension on the 3D Slicer platform and is designed to be accessible to researchers without coding experience. We validate the performance of MEMOS-generated segmentations through comparison to state-of-the-art atlas-based segmentation and quantification of previously reported anatomical abnormalities in a Cbx4 knockout strain. This article has an associated First Person interview with the first author of the paper.

The Liver Tumor Segmentation Benchmark (LiTS)

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

Automated segmentation of lungs and lung tumors in mouse micro-CT scans

Here, we have developed an automated image processing algorithm for segmenting lungs and individual lung tumors in in vivo micro-computed tomography (micro-CT) scans of mouse models of non-small cell lung cancer and lung fibrosis. Over 3000 scans acquired across multiple studies were used to train/validate a 3D U-net lung segmentation model and a Support Vector Machine (SVM) classifier to segment individual lung tumors. The U-net lung segmentation algorithm can be used to estimate changes in soft tissue volume within lungs (primarily tumors and blood vessels), whereas the trained SVM is able to discriminate between tumors and blood vessels and identify individual tumors. The trained segmentation algorithms (1) significantly reduce time required for lung and tumor segmentation, (2) reduce bias and error associated with manual image segmentation, and (3) facilitate identification of individual lung tumors and objective assessment of changes in lung and individual tumor volumes under different experimental conditions.

A fully automated deep learning pipeline for micro-CT-imaging-based densitometry of lung fibrosis murine models

Idiopathic pulmonary fibrosis, the archetype of pulmonary fibrosis (PF), is a chronic lung disease of a poor prognosis, characterized by progressively worsening of lung function. Although histology is still the gold standard for PF assessment in preclinical practice, histological data typically involve less than 1% of total lung volume and are not amenable to longitudinal studies. A miniaturized version of computed tomography (µCT) has been introduced to radiologically examine lung in preclinical murine models of PF. The linear relationship between X-ray attenuation and tissue density allows lung densitometry on total lung volume. However, the huge density changes caused by PF usually require manual segmentation by trained operators, limiting µCT deployment in preclinical routine. Deep learning approaches have achieved state-of-the-art performance in medical image segmentation. In this work, we propose a fully automated deep learning approach to segment right and left lung on µCT imaging and subsequently derive lung densitometry. Our pipeline first employs a convolutional network (CNN) for pre-processing at low-resolution and then a 2.5D CNN for higher-resolution segmentation, combining computational advantage of 2D and ability to address 3D spatial coherence without compromising accuracy. Finally, lungs are divided into compartments based on air content assessed by density. We validated this pipeline on 72 mice with different grades of PF, achieving a Dice score of 0.967 on test set. Our tests demonstrate that this automated tool allows for rapid and comprehensive analysis of µCT scans of PF murine models, thus laying the ground for its wider exploitation in preclinical settings.

High-resolution micro-CT for 3D infarct characterization and segmentation in mice stroke models

Characterization of brain infarct lesions in rodent models of stroke is crucial to assess stroke pathophysiology and therapy outcome. Until recently, the analysis of brain lesions was performed using two techniques: (1) histological methods, such as TTC (Triphenyltetrazolium chloride), a time-consuming and inaccurate process; or (2) MRI imaging, a faster, 3D imaging method, that comes at a high cost. In the last decade, high-resolution micro-CT for 3D sample analysis turned into a simple, fast, and cheaper solution. Here, we successfully describe the application of brain contrasting agents (Osmium tetroxide and inorganic iodine) for high-resolution micro-CT imaging for fine location and quantification of ischemic lesion and edema in mouse preclinical stroke models. We used the intraluminal transient MCAO (Middle Cerebral Artery Occlusion) mouse stroke model to identify and quantify ischemic lesion and edema, and segment core and penumbra regions at different time points after ischemia, by manual and automatic methods. In the transient-ischemic-attack (TIA) mouse model, we can quantify striatal myelinated fibers degeneration. Of note, whole brain 3D reconstructions allow brain atlas co-registration, to identify the affected brain areas, and correlate them with functional impairment. This methodology proves to be a breakthrough in the field, by providing a precise and detailed assessment of stroke outcomes in preclinical animal studies.

Deep learning multi-organ segmentation for whole mouse cryo-images including a comparison of 2D and 3D deep networks

Cryo-imaging provided 3D whole-mouse microscopic color anatomy and fluorescence images that enables biotechnology applications (e.g., stem cells and metastatic cancer). In this report, we compared three methods of organ segmentation: 2D U-Net with 2D-slices and 3D U-Net with either 3D-whole-mouse or 3D-patches. We evaluated the brain, thymus, lung, heart, liver, stomach, spleen, left and right kidney, and bladder. Training with 63 mice, 2D-slices had the best performance, with median Dice scores of > 0.9 and median Hausdorff distances of < 1.2 mm in eightfold cross validation for all organs, except bladder, which is a problem organ due to variable filling and poor contrast. Results were comparable to those for a second analyst on the same data. Regression analyses were performed to fit learning curves, which showed that 2D-slices can succeed with fewer samples. Review and editing of 2D-slices segmentation results reduced human operator time from ~ 2-h to ~ 25-min, with reduced inter-observer variability. As demonstrations, we used organ segmentation to evaluate size changes in liver disease and to quantify the distribution of therapeutic mesenchymal stem cells in organs. With a 48-GB GPU, we determined that extra GPU RAM improved the performance of 3D deep learning because we could train at a higher resolution.

Ultrahigh resolution whole body photon counting computed tomography as a novel versatile tool for translational research from mouse to man

X-ray computed tomography (CT) is a cardinal tool in clinical practice. It provides cross-sectional images within seconds. The recent introduction of clinical photon-counting CT allowed for an increase in spatial resolution by more than a factor of two resulting in a pixel size in the center of rotation of about 150 µm. This level of spatial resolution is in the order of dedicated preclinical micro-CT systems. However so far, the need for different dedicated clinical and preclinical systems often hinders the rapid translation of early research results to applications in men. This drawback might be overcome by ultra-high resolution (UHR) clinical photon-counting CT unifying preclinical and clinical research capabilities in a single machine. Herein, the prototype of a clinical UHR PCD CT (SOMATOM CounT, Siemens Healthineers, Forchheim, Germany) was used. The system comprises a conventional energy-integrating detector (EID) and a novel photon-counting detector (PCD). While the EID provides a pixel size of 0.6 mm in the centre of rotation, the PCD provides a pixel size of 0.25 mm. Additionally, it provides a quantification of photon energies by sorting them into up to four distinct energy bins. This acquisition of multi-energy data allows for a multitude of applications, e.g. pseudo-monochromatic imaging. In particular, we examine the relation between spatial resolution, image noise and administered radiation dose for a multitude of use-cases. These cases include ultra-high resolution and multi-energy acquisitions of mice administered with a prototype bismuth-based contrast agent (nanoPET Pharma, Berlin, Germany) as well as larger animals and actual patients. The clinical EID provides a spatial resolution of about 9 lp/cm (modulation transfer function at 10%, MTF_10%) while UHR allows for the acquisition of images with up to 16 lp/cm allowing for the visualization of all relevant anatomical structures in preclinical and clinical specimen. The spectral capabilities of the system enable a variety of applications previously not available in preclinical research such as pseudo-monochromatic images. Clinical ultra-high resolution photon-counting CT has the potential to unify preclinical and clinical research on a single system enabling versatile imaging of specimens and individuals ranging from mice to man.

Estimating optical parameters of biological tissues with photon-counting micro-CT

Wavelength-dependent absorption and scattering properties determine the fluorescence photon transport in biological tissues and image resolution of optical molecular tomography. Currently, these parameters are computed from optically measured data. For small animal imaging, estimation of optical parameters is a large-scale optimization problem, which is highly ill-posed. In this paper, we propose a new, to the best of our knowledge, approach to estimate optical parameters of biological tissues with photon-counting micro-computed tomography (micro-CT). From photon-counting x-ray data, multi-energy micro-CT images can be reconstructed to perform multi-organ segmentation and material decomposition in terms of tissue constituents. The concentration and characteristics of major tissue constituents can be utilized to calculate the optical absorption and scattering coefficients of the involved tissues. In our study, we perform numerical simulation, phantom experiments, and in vivo animal studies to calculate the optical parameters using our proposed approach. The results show that our approach can estimate optical parameters of tissues with a relative error of <10%, accurately mapping the optical parameter distributions in a small animal.

Deep Learning-based Automatic Lung Segmentation on Multiresolution CT Scans from Healthy and Fibrotic Lungs in Mice

PURPOSE

To develop a model to accurately segment mouse lungs with varying levels of fibrosis and investigate its applicability to mouse images with different resolutions.

MATERIALS AND METHODS

In this experimental retrospective study, a U-Net was trained to automatically segment lungs on mouse CT images. The model was trained (n = 1200), validated (n = 300), and tested (n = 154) on longitudinally acquired and semiautomatically segmented CT images, which included both healthy and irradiated mice (group A). A second independent group of 237 mice (group B) was used for external testing. The Dice score coefficient (DSC) and Hausdorff distance (HD) were used as metrics to quantify segmentation accuracy. Transfer learning was applied to adapt the model to high-spatial-resolution mouse micro-CT segmentation (n = 20; group C [n = 16 for training and n = 4 for testing]).

RESULTS

The trained model yielded a high median DSC in both test datasets: 0.984 (interquartile range [IQR], 0.977-0.988) in group A and 0.966 (IQR, 0.955-0.972) in group B. The median HD in both test datasets was 0.47 mm (IQR, 0-0.51 mm [group A]) and 0.31 mm (IQR, 0.30-0.32 mm [group B]). Spatially resolved quantification of differences toward reference masks revealed two hot spots close to the air-tissue interfaces, which are particularly prone to deviation. Finally, for the higher-resolution mouse CT images, the median DSC was 0.905 (IQR, 0.902-0.929) and the median 95th percentile of the HD was 0.33 mm (IQR, 2.61-2.78 mm).

CONCLUSION

The developed deep learning-based method for mouse lung segmentation performed well independently of disease state (healthy, fibrotic, emphysematous lungs) and CT resolution.Keywords: Deep Learning, Lung Fibrosis, Radiation Therapy, Segmentation, Animal Studies, CT, Thorax, Lung Supplemental material is available for this article. Published under a CC BY 4.0 license.

Reader Perceptions and Impact of AI on CT Assessment of Air Trapping

Quantitative imaging measurements can be facilitated by artificial intelligence (AI) algorithms, but how they might impact decision-making and be perceived by radiologists remains uncertain. After creation of a dedicated inspiratory-expiratory CT examination and concurrent deployment of a quantitative AI algorithm for assessing air trapping, five cardiothoracic radiologists retrospectively evaluated severity of air trapping on 17 examination studies. Air trapping severity of each lobe was evaluated in three stages: qualitatively (visually); semiquantitatively, allowing manual region-of-interest measurements; and quantitatively, using results from an AI algorithm. Readers were surveyed on each case for their perceptions of the AI algorithm. The algorithm improved interreader agreement (intraclass correlation coefficients: visual, 0.28; semiquantitative, 0.40; quantitative, 0.84; P < .001) and improved correlation with pulmonary function testing (forced expiratory volume in 1 second-to-forced vital capacity ratio) (visual r = -0.26, semiquantitative r = -0.32, quantitative r = -0.44). Readers perceived moderate agreement with the AI algorithm (Likert scale average, 3.7 of 5), a mild impact on their final assessment (average, 2.6), and a neutral perception of overall utility (average, 3.5). Though the AI algorithm objectively improved interreader consistency and correlation with pulmonary function testing, individual readers did not immediately perceive this benefit, revealing a potential barrier to clinical adoption. Keywords: Technology Assessment, Quantification © RSNA, 2021.

Virtual monoenergetic micro-CT imaging in mice with artificial intelligence

Micro cone-beam computed tomography (µCBCT) imaging is of utmost importance for carrying out extensive preclinical research in rodents. The imaging of animals is an essential step prior to preclinical precision irradiation, but also in the longitudinal assessment of treatment outcomes. However, imaging artifacts such as beam hardening will occur due to the low energetic nature of the X-ray imaging beam (i.e., 60 kVp). Beam hardening artifacts are especially difficult to resolve in a ‘pancake’ imaging geometry with stationary source and detector, where the animal is rotated around its sagittal axis, and the X-ray imaging beam crosses a wide range of thicknesses. In this study, a seven-layer U-Net based network architecture (vMonoCT) is adopted to predict virtual monoenergetic X-ray projections from polyenergetic X-ray projections. A Monte Carlo simulation model is developed to compose a training dataset of 1890 projection pairs. Here, a series of digital anthropomorphic mouse phantoms was derived from the reference DigiMouse phantom as simulation geometry. vMonoCT was trained on 1512 projection pairs (= 80%) and tested on 378 projection pairs (= 20%). The percentage error calculated for the test dataset was 1.7 ± 0.4%. Additionally, the vMonoCT model was evaluated on a retrospective projection dataset of five mice and one frozen cadaver. It was found that beam hardening artifacts were minimized after image reconstruction of the vMonoCT-corrected projections, and that anatomically incorrect gradient errors were corrected in the cranium up to 15%. Our results disclose the potential of Artificial Intelligence to enhance the µCBCT image quality in biomedical applications. vMonoCT is expected to contribute to the reproducibility of quantitative preclinical applications such as precision irradiations in X-ray cabinets, and to the evaluation of longitudinal imaging data in extensive preclinical studies.

Automatic contouring of normal tissues with deep learning for preclinical radiation studies

Objective.Delineation of relevant normal tissues is a bottleneck in image-guided precision radiotherapy workflows for small animals. A deep learning (DL) model for automatic contouring using standardized 3D micro cone-beam CT (μCBCT) volumes as input is proposed, to provide a fully automatic, generalizable method for normal tissue contouring in preclinical studies.Approach.A 3D U-net was trained to contour organs in the head (whole brain, left/right brain hemisphere, left/right eye) and thorax (complete lungs, left/right lung, heart, spinal cord, thorax bone) regions. As an important preprocessing step, Hounsfield units (HUs) were converted to mass density (MD) values, to remove the energy dependency of theμCBCT scanner and improve generalizability of the DL model. Model performance was evaluated quantitatively by Dice similarity coefficient (DSC), mean surface distance (MSD), 95th percentile Hausdorff distance (HD_95p), and center of mass displacement (ΔCoM). For qualitative assessment, DL-generated contours (for 40 and 80 kV images) were scored (0: unacceptable, manual re-contouring needed - 5: no adjustments needed). An uncertainty analysis using Monte Carlo dropout uncertainty was performed for delineation of the heart.Main results.The proposed DL model and accompanying preprocessing method provide high quality contours, with in general median DSC > 0.85, MSD < 0.25 mm, HD_95p < 1 mm and ΔCoM < 0.5 mm. The qualitative assessment showed very few contours needed manual adaptations (40 kV: 20/155 contours, 80 kV: 3/155 contours). The uncertainty of the DL model is small (within 2%).Significance.A DL-based model dedicated to preclinical studies has been developed for multi-organ segmentation in two body sites. For the first time, a method independent of image acquisition parameters has been quantitatively evaluated, resulting in sub-millimeter performance, while qualitative assessment demonstrated the high quality of the DL-generated contours. The uncertainty analysis additionally showed that inherent model variability is low.

Deep learning-based segmentation of the thorax in mouse micro-CT scans

For image-guided small animal irradiations, the whole workflow of imaging, organ contouring, irradiation planning, and delivery is typically performed in a single session requiring continuous administration of anaesthetic agents. Automating contouring leads to a faster workflow, which limits exposure to anaesthesia and thereby, reducing its impact on experimental results and on animal wellbeing. Here, we trained the 2D and 3D U-Net architectures of no-new-Net (nnU-Net) for autocontouring of the thorax in mouse micro-CT images. We trained the models only on native CTs and evaluated their performance using an independent testing dataset (i.e., native CTs not included in the training and validation). Unlike previous studies, we also tested the model performance on an external dataset (i.e., contrast-enhanced CTs) to see how well they predict on CTs completely different from what they were trained on. We also assessed the interobserver variability using the generalized conformity index ([Formula: see text]) among three observers, providing a stronger human baseline for evaluating automated contours than previous studies. Lastly, we showed the benefit on the contouring time compared to manual contouring. The results show that 3D models of nnU-Net achieve superior segmentation accuracy and are more robust to unseen data than 2D models. For all target organs, the mean surface distance (MSD) and the Hausdorff distance (95p HD) of the best performing model for this task (nnU-Net 3d_fullres) are within 0.16 mm and 0.60 mm, respectively. These values are below the minimum required contouring accuracy of 1 mm for small animal irradiations, and improve significantly upon state-of-the-art 2D U-Net-based AIMOS method. Moreover, the conformity indices of the 3d_fullres model also compare favourably to the interobserver variability for all target organs, whereas the 2D models perform poorly in this regard. Importantly, the 3d_fullres model offers 98% reduction in contouring time.

Inter-observer variability of organ contouring for preclinical studies with cone beam Computed Tomography imaging

Background and purpose

In preclinical radiation studies, there is great interest in quantifying the radiation response of healthy tissues. Manual contouring has significant impact on the treatment-planning because of variation introduced by human interpretation. This results in inconsistencies when assessing normal tissue volumes. Evaluation of these discrepancies can provide a better understanding on the limitations of the current preclinical radiation workflow. In the present work, interobserver variability (IOV) in manual contouring of rodent normal tissues on cone-beam Computed Tomography, in head and thorax regions was evaluated.

Materials and methods

Two animal technicians performed manually (assisted) contouring of normal tissues located within the thorax and head regions of rodents, 20 cases per body site. Mean surface distance (MSD), displacement of center of mass (ΔCoM), DICE similarity coefficient (DSC) and the 95th percentile Hausdorff distance (HD₉₅) were calculated between the contours of the two observers to evaluate the IOV.

Results

For the thorax organs, right lung had the lowest IOV (ΔCoM: 0.08 ± 0.04 mm, DSC: 0.96 ± 0.01, MSD:0.07 ± 0.01 mm, HD₉₅:0.20 ± 0.03 mm) while spinal cord, the highest IOV (ΔCoM:0.5 ± 0.3 mm, DSC:0.81 ± 0.05, MSD:0.14 ± 0.03 mm, HD₉₅:0.8 ± 0.2 mm). Regarding head organs, right eye demonstrated the lowest IOV (ΔCoM:0.12 ± 0.08 mm, DSC: 0.93 ± 0.02, MSD: 0.15 ± 0.04 mm, HD₉₅: 0.29 ± 0.07 mm) while complete brain, the highest IOV (ΔCoM: 0.2 ± 0.1 mm, DSC: 0.94 ± 0.02, MSD: 0.3 ± 0.1 mm, HD₉₅: 0.5 ± 0.1 mm).

Conclusions

Our findings reveal small IOV, within the sub-mm range, for thorax and head normal tissues in rodents. The set of contours can serve as a basis for developing an automated delineation method for e.g., treatment planning.

TISSUE CLEARING

Tissue clearing of gross anatomical samples was first described over a century ago and has only recently found widespread use in the field of microscopy. This renaissance has been driven by the application of modern knowledge of optical physics and chemical engineering to the development of robust and reproducible clearing techniques, the arrival of new microscopes that can image large samples at cellular resolution and computing infrastructure able to store and analyze large data volumes. Many biological relationships between structure and function require investigation in three dimensions and tissue clearing therefore has the potential to enable broad discoveries in the biological sciences. Unfortunately, the current literature is complex and could confuse researchers looking to begin a clearing project. The goal of this Primer is to outline a modular approach to tissue clearing that allows a novice researcher to develop a customized clearing pipeline tailored to their tissue of interest. Further, the Primer outlines the required imaging and computational infrastructure needed to perform tissue clearing at scale, gives an overview of current applications, discusses limitations and provides an outlook on future advances in the field.

A study of generalization and compatibility performance of 3D U-Net segmentation on multiple heterogeneous liver CT datasets

BACKGROUND

Most existing algorithms have been focused on the segmentation from several public Liver CT datasets scanned regularly (no pneumoperitoneum and horizontal supine position). This study primarily segmented datasets with unconventional liver shapes and intensities deduced by contrast phases, irregular scanning conditions, different scanning objects of pigs and patients with large pathological tumors, which formed the multiple heterogeneity of datasets used in this study.

METHODS

The multiple heterogeneous datasets used in this paper includes: (1) One public contrast-enhanced CT dataset and one public non-contrast CT dataset; (2) A contrast-enhanced dataset that has abnormal liver shape with very long left liver lobes and large-sized liver tumors with abnormal presets deduced by microvascular invasion; (3) One artificial pneumoperitoneum dataset under the pneumoperitoneum and three scanning profiles (horizontal/left/right recumbent position); (4) Two porcine datasets of Bama type and domestic type that contains pneumoperitoneum cases but with large anatomy discrepancy with humans. The study aimed to investigate the segmentation performances of 3D U-Net in: (1) generalization ability between multiple heterogeneous datasets by cross-testing experiments; (2) the compatibility when hybrid training all datasets in different sampling and encoder layer sharing schema. We further investigated the compatibility of encoder level by setting separate level for each dataset (i.e., dataset-wise convolutions) while sharing the decoder.

RESULTS

Model trained on different datasets has different segmentation performance. The prediction accuracy between LiTS dataset and Zhujiang dataset was about 0.955 and 0.958 which shows their good generalization ability due to that they were all contrast-enhanced clinical patient datasets scanned regularly. For the datasets scanned under pneumoperitoneum, their corresponding datasets scanned without pneumoperitoneum showed good generalization ability. Dataset-wise convolution module in high-level can improve the dataset unbalance problem. The experimental results will facilitate researchers making solutions when segmenting those special datasets.

CONCLUSIONS

(1) Regularly scanned datasets is well generalized to irregularly ones. (2) The hybrid training is beneficial but the dataset imbalance problem always exits due to the multi-domain homogeneity. The higher levels encoded more domain specific information than lower levels and thus were less compatible in terms of our datasets.

Tissue clearing to examine tumour complexity in three dimensions

The visualization of whole organs and organisms through tissue clearing and fluorescence volumetric imaging has revolutionized the way we look at biological samples. Its application to solid tumours is changing our perception of tumour architecture, revealing signalling networks and cell interactions critical in tumour progression, and provides a powerful new strategy for cancer diagnostics. This Review introduces the latest advances in tissue clearing and three-dimensional imaging, examines the challenges in clearing epithelia - the tissue of origin of most malignancies - and discusses the insights that tissue clearing has brought to cancer research, as well as the prospective applications to experimental and clinical oncology.

Deep-Learning-Based Automatic Segmentation of Head and Neck Organs for Radiation Therapy in Dogs

Purpose: This study was conducted to develop a deep learning-based automatic segmentation (DLBAS) model of head and neck organs for radiotherapy (RT) in dogs, and to evaluate the feasibility for delineating the RT planning.

Materials and Methods: The segmentation indicated that there were potentially 15 organs at risk (OARs) in the head and neck of dogs. Post-contrast computed tomography (CT) was performed in 90 dogs. The training and validation sets comprised 80 CT data sets, including 20 test sets. The accuracy of the segmentation was assessed using both the Dice similarity coefficient (DSC) and the Hausdorff distance (HD), and by referencing the expert contours as the ground truth. An additional 10 clinical test sets with relatively large displacement or deformation of organs were selected for verification in cancer patients. To evaluate the applicability in cancer patients, and the impact of expert intervention, three methods–HA, DLBAS, and the readjustment of the predicted data obtained via the DLBAS of the clinical test sets (HA_DLBAS)–were compared.

Results: The DLBAS model (in the 20 test sets) showed reliable DSC and HD values; it also had a short contouring time of ~3 s. The average (mean ± standard deviation) DSC (0.83 ± 0.04) and HD (2.71 ± 1.01 mm) values were similar to those of previous human studies. The DLBAS was highly accurate and had no large displacement of head and neck organs. However, the DLBAS in the 10 clinical test sets showed lower DSC (0.78 ± 0.11) and higher HD (4.30 ± 3.69 mm) values than those of the test sets. The HA_DLBAS was comparable to both the HA (DSC: 0.85 ± 0.06 and HD: 2.74 ± 1.18 mm) and DLBAS presented better comparison metrics and decreased statistical deviations (DSC: 0.94 ± 0.03 and HD: 2.30 ± 0.41 mm). In addition, the contouring time of HA_DLBAS (30 min) was less than that of HA (80 min).

Conclusion: In conclusion, HA_DLBAS method and the proposed DLBAS was highly consistent and robust in its performance. Thus, DLBAS has great potential as a single or supportive tool to the key process in RT planning.