Automatic Multi-Organ Segmentation on Abdominal CT With Dense V-Networks

Deep Learning Techniques for Medical Image Segmentation: Achievements and Challenges

Deep learning-based image segmentation is by now firmly established as a robust tool in image segmentation. It has been widely used to separate homogeneous areas as the first and critical component of diagnosis and treatment pipeline. In this article, we present a critical appraisal of popular methods that have employed deep-learning techniques for medical image segmentation. Moreover, we summarize the most common challenges incurred and suggest possible solutions.

Deep Q Learning Driven CT Pancreas Segmentation With Geometry-Aware U-Net

The segmentation of pancreas is important for medical image analysis, yet it faces great challenges of class imbalance, background distractions, and non-rigid geometrical features. To address these difficulties, we introduce a deep Q network (DQN) driven approach with deformable U-Net to accurately segment the pancreas by explicitly interacting with contextual information and extract anisotropic features from pancreas. The DQN-based model learns a context-adaptive localization policy to produce a visually tightened and precise localization bounding box of the pancreas. Furthermore, deformable U-Net captures geometry-aware information of pancreas by learning geometrically deformable filters for feature extraction. The experiments on NIH dataset validate the effectiveness of the proposed framework in pancreas segmentation.

Annotated normal CT data of the abdomen for deep learning: Challenges and strategies for implementation

PURPOSE

The purpose of this study was to report procedures developed to annotate abdominal computed tomography (CT) images from subjects without pancreatic disease that will be used as the input for deep convolutional neural networks (DNN) for development of deep learning algorithms for automatic recognition of a normal pancreas.

MATERIALS AND METHODS

Dual-phase contrast-enhanced volumetric CT acquired from 2005 to 2009 from potential kidney donors were retrospectively assessed. Four trained human annotators manually and sequentially annotated 22 structures in each datasets, then expert radiologists confirmed the annotation. For efficient annotation and data management, a commercial software package that supports three-dimensional segmentation was used.

RESULTS

A total of 1150 dual-phase CT datasets from 575 subjects were annotated. There were 229 men and 346 women (mean age: 45±12years; range: 18-79years). The mean intra-observer intra-subject dual-phase CT volume difference of all annotated structures was 4.27mL (7.65%). The deep network prediction for multi-organ segmentation showed high fidelity with 89.4% and 1.29mm in terms of mean Dice similarity coefficients and mean surface distances, respectively.

CONCLUSIONS

A reliable data collection/annotation process for abdominal structures was developed. This process can be used to generate large datasets appropriate for deep learning.

Deep learning application engine (DLAE): Development and integration of deep learning algorithms in medical imaging

Herein we introduce a deep learning (DL) application engine (DLAE) system concept, present potential uses of it, and describe pathways for its integration in clinical workflows. An open-source software application was developed to provide a code-free approach to DL for medical imaging applications. DLAE supports several DL techniques used in medical imaging, including convolutional neural networks, fully convolutional networks, generative adversarial networks, and bounding box detectors. Several example applications using clinical images were developed and tested to demonstrate the capabilities of DLAE. Additionally, a model deployment example was demonstrated in which DLAE was used to integrate two trained models into a commercial clinical software package.

Preprocessing of Medical Image Data for Three-Dimensional Bioprinted Customized-Neural-Scaffolds

IMPACT STATEMENT

Nerve damage, which can be devastating, triggers several biological cascades, which result in the insufficiencies of the human nervous system to provide complete nerve repair and regain of function. Since no therapeutic strategy exists to provide immediate attention and intervention to patients with newly acquired nerve damage, we propose a strategy in which accelerated medical image processing through graphical processing unit implementation and three-dimensional printing are combined to produce a time-efficient, patient-specific (custom-neural-scaffold) solution to nerve damage. This work aims to beneficially shorten the time required for medical decision-making so that improved patient outcomes are achieved.

Advances in Auto-Segmentation

Manual image segmentation is a time-consuming task routinely performed in radiotherapy to identify each patient's targets and anatomical structures. The efficacy and safety of the radiotherapy plan requires accurate segmentations as these regions of interest are generally used to optimize and assess the quality of the plan. However, reports have shown that this process can be subject to significant inter- and intraobserver variability. Furthermore, the quality of the radiotherapy treatment, and subsequent analyses (ie, radiomics, dosimetric), can be subject to the accuracy of these manual segmentations. Automatic segmentation (or auto-segmentation) of targets and normal tissues is, therefore, preferable as it would address these challenges. Previously, auto-segmentation techniques have been clustered into 3 generations of algorithms, with multiatlas based and hybrid techniques (third generation) being considered the state-of-the-art. More recently, however, the field of medical image segmentation has seen accelerated growth driven by advances in computer vision, particularly through the application of deep learning algorithms, suggesting we have entered the fourth generation of auto-segmentation algorithm development. In this paper, the authors review traditional (nondeep learning) algorithms particularly relevant for applications in radiotherapy. Concepts from deep learning are introduced focusing on convolutional neural networks and fully-convolutional networks which are generally used for segmentation tasks. Furthermore, the authors provide a summary of deep learning auto-segmentation radiotherapy applications reported in the literature. Lastly, considerations for clinical deployment (commissioning and QA) of auto-segmentation software are provided.

Fast and robust active neuron segmentation in two-photon calcium imaging using spatiotemporal deep learning

Calcium imaging records large-scale neuronal activity with cellular resolution in vivo. Automated, fast, and reliable active neuron segmentation is a critical step in the analysis workflow of utilizing neuronal signals in real-time behavioral studies for discovery of neuronal coding properties. Here, to exploit the full spatiotemporal information in two-photon calcium imaging movies, we propose a 3D convolutional neural network to identify and segment active neurons. By utilizing a variety of two-photon microscopy datasets, we show that our method outperforms state-of-the-art techniques and is on a par with manual segmentation. Furthermore, we demonstrate that the network trained on data recorded at a specific cortical layer can be used to accurately segment active neurons from another layer with different neuron density. Finally, our work documents significant tabulation flaws in one of the most cited and active online scientific challenges in neuron segmentation. As our computationally fast method is an invaluable tool for a large spectrum of real-time optogenetic experiments, we have made our open-source software and carefully annotated dataset freely available online.

A novel biomarker, active whole skeletal total lesion glycolysis (WS-TLG), as a quantitative method to measure bone metastatic activity in breast cancer patients

Objective

There is no good response evaluation method for skeletal metastasis. We aimed to develop a novel quantitative method to evaluate the response of skeletal metastasis, especially lytic lesions, for treatment.

Methods

A method to measure active bone metastatic burden quantitatively using F-18 fluorodeoxyglucose positron emission tomography with computed tomography (FDG–PET/CT) in breast cancer patients, whole skeletal total lesion glycolysis (WS-TLG), a summation of each skeletal lesion’s TLG, was developed. To identify active bone lesions, a tentative cutoff value was decided using FDG–PET/CT in 85 breast cancer patients without skeletal metastasis and 35 with skeletal metastasis by changing the cutoff value. Then, the WS-TLG method was evaluated by comparing to PET Response Criteria in Solid Tumor (PERCIST) or European Organization for Research and Treatment of Cancer (EORTC) criteria for only bone in 15 breast cancer patients with skeletal metastasis who were treated.

Results

A cutoff value of the standardized uptake value (SUV) = 4.0 gave 91% (77/85) specificity and 97% (34/35) sensitivity. We decided on SUV = 4.0 as a tentative cutoff value. Skeletal metastases of lytic and mixed types showed higher WS-TLG values than those of blastic or intertrabecular types, although statistical significance was not tested. All 15 patients showed agreement with PERCIST or EORTC in the therapeutic bone response.

Conclusion

This quantitative WS-TLG method appears to be a good biomarker to evaluate skeletal metastasis in breast cancer patients, especially lytic or mixed types. Further clinical studies are warranted to assess the clinical values of this new WS-TLG method.

What the radiologist should know about artificial intelligence - an ESR white paper

This paper aims to provide a review of the basis for application of AI in radiology, to discuss the immediate ethical and professional impact in radiology, and to consider possible future evolution.Even if AI does add significant value to image interpretation, there are implications outside the traditional radiology activities of lesion detection and characterisation. In radiomics, AI can foster the analysis of the features and help in the correlation with other omics data. Imaging biobanks would become a necessary infrastructure to organise and share the image data from which AI models can be trained. AI can be used as an optimising tool to assist the technologist and radiologist in choosing a personalised patient's protocol, tracking the patient's dose parameters, providing an estimate of the radiation risks. AI can also aid the reporting workflow and help the linking between words, images, and quantitative data. Finally, AI coupled with CDS can improve the decision process and thereby optimise clinical and radiological workflow.

Precision Medicine in Pancreatic Disease-Knowledge Gaps and Research Opportunities: Summary of a National Institute of Diabetes and Digestive and Kidney Diseases Workshop

A workshop on research gaps and opportunities for Precision Medicine in Pancreatic Disease was sponsored by the National Institute of Diabetes and Digestive Kidney Diseases on July 24, 2019, in Pittsburgh. The workshop included an overview lecture on precision medicine in cancer and 4 sessions: (1) general considerations for the application of bioinformatics and artificial intelligence; (2) omics, the combination of risk factors and biomarkers; (3) precision imaging; and (4) gaps, barriers, and needs to move from precision to personalized medicine for pancreatic disease. Current precision medicine approaches and tools were reviewed, and participants identified knowledge gaps and research needs that hinder bringing precision medicine to pancreatic diseases. Most critical were (a) multicenter efforts to collect large-scale patient data sets from multiple data streams in the context of environmental and social factors; (b) new information systems that can collect, annotate, and quantify data to inform disease mechanisms; (c) novel prospective clinical trial designs to test and improve therapies; and (d) a framework for measuring and assessing the value of proposed approaches to the health care system. With these advances, precision medicine can identify patients early in the course of their pancreatic disease and prevent progression to chronic or fatal illness.

Augmented visualization with depth perception cues to improve the surgeon's performance in minimally invasive surgery

Minimally invasive techniques, such as laparoscopy and radiofrequency ablation of tumors, bring important advantages in surgery: by minimizing incisions on the patient's body, they can reduce the hospitalization period and the risk of postoperative complications. Unfortunately, they come with drawbacks for surgeons, who have a restricted vision of the operation area through an indirect access and 2D images provided by a camera inserted in the body. Augmented reality provides an "X-ray vision" of the patient anatomy thanks to the visualization of the internal organs of the patient. In this way, surgeons are free from the task of mentally associating the content from CT images to the operative scene. We present a navigation system that supports surgeons in preoperative and intraoperative phases and an augmented reality system that superimposes virtual organs on the patient's body together with depth and distance information. We implemented a combination of visual and audio cues allowing the surgeon to improve the intervention precision and avoid the risk of damaging anatomical structures. The test scenarios proved the good efficacy and accuracy of the system. Moreover, tests in the operating room suggested some modifications to the tracking system to make it more robust with respect to occlusions. Graphical Abstract Augmented visualization in minimally invasive surgery.