Self-channel-and-spatial-attention neural network for automated multi-organ segmentation on head and neck CT images

Applications and challenges of neural networks in otolaryngology (Review)

Artificial Intelligence (AI) has become a topic of interest that is frequently debated in all research fields. The medical field is no exception, where several unanswered questions remain. When and how this field can benefit from AI support in daily routines are the most frequently asked questions. The present review aims to present the types of neural networks (NNs) available for development, discussing their advantages, disadvantages and how they can be applied practically. In addition, the present review summarizes how NNs (combined with various other features) have already been applied in studies in the ear nose throat research field, from assisting diagnosis to treatment management. Although the answer to this question regarding AI remains elusive, understanding the basics and types of applicable NNs can lead to future studies possibly using more than one type of NN. This approach may bypass the actual limitations in accuracy and relevance of information generated by AI. The proposed studies, the majority of which used convolutional NNs, obtained accuracies varying 70-98%, with a number of studies having the AI trained on a limited number of cases (<100 patients). The lack of standardization in AI protocols for research negatively affects data homogeneity and transparency of databases.

Multi-organ segmentation: a progressive exploration of learning paradigms under scarce annotation

Precise delineation of multiple organs or abnormal regions in the human body from medical images plays an essential role in computer-aided diagnosis, surgical simulation, image-guided interventions, and especially in radiotherapy treatment planning. Thus, it is of great significance to explore automatic segmentation approaches, among which deep learning-based approaches have evolved rapidly and witnessed remarkable progress in multi-organ segmentation. However, obtaining an appropriately sized and fine-grained annotated dataset of multiple organs is extremely hard and expensive. Such scarce annotation limits the development of high-performance multi-organ segmentation models but promotes many annotation-efficient learning paradigms. Among these, studies on transfer learning leveraging external datasets, semi-supervised learning including unannotated datasets and partially-supervised learning integrating partially-labeled datasets have led the dominant way to break such dilemmas in multi-organ segmentation. We first review the fully supervised method, then present a comprehensive and systematic elaboration of the 3 abovementioned learning paradigms in the context of multi-organ segmentation from both technical and methodological perspectives, and finally summarize their challenges and future trends.

Deep learning algorithm performance in contouring head and neck organs at risk: a systematic review and single-arm meta-analysis

PURPOSE

The contouring of organs at risk (OARs) in head and neck cancer radiation treatment planning is a crucial, yet repetitive and time-consuming process. Recent studies have applied deep learning (DL) algorithms to automatically contour head and neck OARs. This study aims to conduct a systematic review and meta-analysis to summarize and analyze the performance of DL algorithms in contouring head and neck OARs. The objective is to assess the advantages and limitations of DL algorithms in contour planning of head and neck OARs.

METHODS

This study conducted a literature search of Pubmed, Embase and Cochrane Library databases, to include studies related to DL contouring head and neck OARs, and the dice similarity coefficient (DSC) of four categories of OARs from the results of each study are selected as effect sizes for meta-analysis. Furthermore, this study conducted a subgroup analysis of OARs characterized by image modality and image type.

RESULTS

149 articles were retrieved, and 22 studies were included in the meta-analysis after excluding duplicate literature, primary screening, and re-screening. The combined effect sizes of DSC for brainstem, spinal cord, mandible, left eye, right eye, left optic nerve, right optic nerve, optic chiasm, left parotid, right parotid, left submandibular, and right submandibular are 0.87, 0.83, 0.92, 0.90, 0.90, 0.71, 0.74, 0.62, 0.85, 0.85, 0.82, and 0.82, respectively. For subgroup analysis, the combined effect sizes for segmentation of the brainstem, mandible, left optic nerve, and left parotid gland using CT and MRI images are 0.86/0.92, 0.92/0.90, 0.71/0.73, and 0.84/0.87, respectively. Pooled effect sizes using 2D and 3D images of the brainstem, mandible, left optic nerve, and left parotid gland for contouring are 0.88/0.87, 0.92/0.92, 0.75/0.71 and 0.87/0.85.

CONCLUSIONS

The use of automated contouring technology based on DL algorithms is an essential tool for contouring head and neck OARs, achieving high accuracy, reducing the workload of clinical radiation oncologists, and providing individualized, standardized, and refined treatment plans for implementing "precision radiotherapy". Improving DL performance requires the construction of high-quality data sets and enhancing algorithm optimization and innovation.

MHL-Net: A Multistage Hierarchical Learning Network for Head and Neck Multiorgan Segmentation

Accurate segmentation of head and neck organs at risk is crucial in radiotherapy. However, the existing methods suffer from incomplete feature mining, insufficient information utilization, and difficulty in simultaneously improving the performance of small and large organ segmentation. In this paper, a multistage hierarchical learning network is designed to fully extract multidimensional features, combined with anatomical prior information and imaging features, using multistage subnetworks to improve the segmentation performance. First, multilevel subnetworks are constructed for primary segmentation, localization, and fine segmentation by dividing organs into two levels-large and small. Different networks both have their own learning focuses and feature reuse and information sharing among each other, which comprehensively improved the segmentation performance of all organs. Second, an anatomical prior probability map and a boundary contour attention mechanism are developed to address the problem of complex anatomical shapes. Prior information and boundary contour features effectively assist in detecting and segmenting special shapes. Finally, a multidimensional combination attention mechanism is proposed to analyze axial, coronal, and sagittal information, capture spatial and channel features, and maximize the use of structural information and semantic features of 3D medical images. Experimental results on several datasets showed that our method was competitive with state-of-the-art methods and improved the segmentation results for multiscale organs.

Validation of clinical acceptability of deep-learning-based automated segmentation of organs-at-risk for head-and-neck radiotherapy treatment planning

Introduction

Organ-at-risk segmentation for head and neck cancer radiation therapy is a complex and time-consuming process (requiring up to 42 individual structure, and may delay start of treatment or even limit access to function-preserving care. Feasibility of using a deep learning (DL) based autosegmentation model to reduce contouring time without compromising contour accuracy is assessed through a blinded randomized trial of radiation oncologists (ROs) using retrospective, de-identified patient data.

Methods

Two head and neck expert ROs used dedicated time to create gold standard (GS) contours on computed tomography (CT) images. 445 CTs were used to train a custom 3D U-Net DL model covering 42 organs-at-risk, with an additional 20 CTs were held out for the randomized trial. For each held-out patient dataset, one of the eight participant ROs was randomly allocated to review and revise the contours produced by the DL model, while another reviewed contours produced by a medical dosimetry assistant (MDA), both blinded to their origin. Time required for MDAs and ROs to contour was recorded, and the unrevised DL contours, as well as the RO-revised contours by the MDAs and DL model were compared to the GS for that patient.

Results

Mean time for initial MDA contouring was 2.3 hours (range 1.6-3.8 hours) and RO-revision took 1.1 hours (range, 0.4-4.4 hours), compared to 0.7 hours (range 0.1-2.0 hours) for the RO-revisions to DL contours. Total time reduced by 76% (95%-Confidence Interval: 65%-88%) and RO-revision time reduced by 35% (95%-CI,-39%-91%). All geometric and dosimetric metrics computed, agreement with GS was equivalent or significantly greater (p&lt;0.05) for RO-revised DL contours compared to the RO-revised MDA contours, including volumetric Dice similarity coefficient (VDSC), surface DSC, added path length, and the 95%-Hausdorff distance. 32 OARs (76%) had mean VDSC greater than 0.8 for the RO-revised DL contours, compared to 20 (48%) for RO-revised MDA contours, and 34 (81%) for the unrevised DL OARs.

Conclusion

DL autosegmentation demonstrated significant time-savings for organ-at-risk contouring while improving agreement with the institutional GS, indicating comparable accuracy of DL model. Integration into the clinical practice with a prospective evaluation is currently underway.

An automatic segmentation method with self-attention mechanism on left ventricle in gated PET/CT myocardial perfusion imaging

OBJECTIVES

We aimed to propose an automatic segmentation method for left ventricular (LV) from 16 electrocardiogram (ECG) -gated ¹³N-NH₃ PET/CT myocardial perfusion imaging (MPI) to improve the performance of LV function assessment.

METHODS

Ninety-six cases with confirmed or suspected obstructive coronary artery disease (CAD) were enrolled in this research. The LV myocardial contours were delineated by physicians as ground truth. We developed an automatic segmentation method, which introduces the self-attention mechanism into 3D U-Net to capture global information of images so as to achieve fine segmentation of LV. Three cross-validation tests were performed on each gate (64 vs. 32 for training vs. validation). The effectiveness was validated by quantitative metrics (modified hausdorff distance, MHD; dice ratio, DR; 3D MHD) as well as cardiac functional parameters (end-systolic volume, ESV; end-diastolic volume, EDV; ejection fraction, EF). Furthermore, the feasibility of the proposed method was also evaluated by intra- and inter-observers with DR and 3D-MHD.

RESULTS

Compared with backbone network, the proposed approach improved the average DR from 0.905 ± 0.0193 to 0.9202 ± 0.0164, and decreased the average 3D MHD from 0.4611 ± 0.0349 to 0.4304 ± 0.0339. The average relative error of LV volume between proposed method and ground truth is 1.09±3.66%, and the correlation coefficient is 0.992 ± 0.007 (P < 0.001). The EDV, ESV, EF deduced from the proposed approach were highly correlated with ground truth (r ≥ 0.864, P < 0.001), and the correlation with commercial software is fair (r ≥ 0.871, P < 0.001). DR and 3D MHD of contours and myocardium from two observers are higher than 0.899 and less than 0.5194.

CONCLUSION

The proposed approach is highly feasible for automatic segmentation of the LV cavity and myocardium, with potential to benefit the precision of LV function assessment.

Mandible segmentation from CT data for virtual surgical planning using an augmented two-stepped convolutional neural network

PURPOSE

For computer-aided planning of facial bony surgery, the creation of high-resolution 3D-models of the bones by segmenting volume imaging data is a labor-intensive step, especially as metal dental inlays or implants cause severe artifacts that reduce the quality of the computer-tomographic imaging data. This study provides a method to segment accurate, artifact-free 3D surface models of mandibles from CT data using convolutional neural networks.

METHODS

The presented approach cascades two independently trained 3D-U-Nets to perform accurate segmentations of the mandible bone from full resolution CT images. The networks are trained in different settings using three different loss functions and a data augmentation pipeline. Training and evaluation datasets consist of manually segmented CT images from 307 dentate and edentulous individuals, partly with heavy imaging artifacts. The accuracy of the models is measured using overlap-based, surface-based and anatomical-curvature-based metrics.

RESULTS

Our approach produces high-resolution segmentations of the mandibles, coping with severe imaging artifacts in the CT imaging data. The use of the two-stepped approach yields highly significant improvements to the prediction accuracies. The best models achieve a Dice coefficient of 94.824% and an average surface distance of 0.31 mm on our test dataset.

CONCLUSION

The use of two cascaded U-Net allows high-resolution predictions for small regions of interest in the imaging data. The proposed method is fast and allows a user-independent image segmentation, producing objective and repeatable results that can be used in automated surgical planning procedures.

Multi-scale contextual semantic enhancement network for 3D medical image segmentation

Objective. Accurate and automatic segmentation of medical images is crucial for improving the efficiency of disease diagnosis and making treatment plans. Although methods based on convolutional neural networks have achieved excellent results in numerous segmentation tasks of medical images, they still suffer from challenges including drastic scale variations of lesions, blurred boundaries of lesions and class imbalance. Our objective is to design a segmentation framework named multi-scale contextual semantic enhancement network (3D MCSE-Net) to address the above problems.Approach. The 3D MCSE-Net mainly consists of a multi-scale context pyramid fusion module (MCPFM), a triple feature adaptive enhancement module (TFAEM), and an asymmetric class correction loss (ACCL) function. Specifically, the MCPFM resolves the problem of unreliable predictions due to variable morphology and drastic scale variations of lesions by capturing the multi-scale global context of feature maps. Subsequently, the TFAEM overcomes the problem of blurred boundaries of lesions caused by the infiltrating growth and complex context of lesions by adaptively recalibrating and enhancing the multi-dimensional feature representation of suspicious regions. Moreover, the ACCL alleviates class imbalances by adjusting asy mmetric correction coefficient and weighting factor.Main results. Our method is evaluated on the nasopharyngeal cancer tumor segmentation (NPCTS) dataset, the public dataset of the MICCAI 2017 liver tumor segmentation (LiTS) challenge and the 3D image reconstruction for comparison of algorithm and DataBase (3Dircadb) dataset to verify its effectiveness and generalizability. The experimental results show the proposed components all have unique strengths and exhibit mutually reinforcing properties. More importantly, the proposed 3D MCSE-Net outperforms previous state-of-the-art methods for tumor segmentation on the NPCTS, LiTS and 3Dircadb dataset.Significance. Our method addresses the effects of drastic scale variations of lesions, blurred boundaries of lesions and class imbalance, and improves tumors segmentation accuracy, which facilitates clinical medical diagnosis and treatment planning.

A review on AI-based medical image computing in head and neck surgery

Head and neck surgery is a fine surgical procedure with a complex anatomical space, difficult operation and high risk. Medical image computing (MIC) that enables accurate and reliable preoperative planning is often needed to reduce the operational difficulty of surgery and to improve patient survival. At present, artificial intelligence, especially deep learning, has become an intense focus of research in MIC. In this study, the application of deep learning-based MIC in head and neck surgery is reviewed. Relevant literature was retrieved on the Web of Science database from January 2015 to May 2022, and some papers were selected for review from mainstream journals and conferences, such as IEEE Transactions on Medical Imaging, Medical Image Analysis, Physics in Medicine and Biology, Medical Physics, MICCAI, etc. Among them, 65 references are on automatic segmentation, 15 references on automatic landmark detection, and eight references on automatic registration. In the elaboration of the review, first, an overview of deep learning in MIC is presented. Then, the application of deep learning methods is systematically summarized according to the clinical needs, and generalized into segmentation, landmark detection and registration of head and neck medical images. In segmentation, it is mainly focused on the automatic segmentation of high-risk organs, head and neck tumors, skull structure and teeth, including the analysis of their advantages, differences and shortcomings. In landmark detection, the focus is mainly on the introduction of landmark detection in cephalometric and craniomaxillofacial images, and the analysis of their advantages and disadvantages. In registration, deep learning networks for multimodal image registration of the head and neck are presented. Finally, their shortcomings and future development directions are systematically discussed. The study aims to serve as a reference and guidance for researchers, engineers or doctors engaged in medical image analysis of head and neck surgery.

Optimising a 3D convolutional neural network for head and neck computed tomography segmentation with limited training data

•

Convolutional neural networks (CNNs) are used for auto-segmentation in radiotherapy.

•

However, CNNs rely on large, high-quality datasets: a scarcity in radiotherapy.

•

We develop a CNN model, trained with limited data, for accurate segmentation.

•

Multiple experiments were performed to optimise key features of our custom model.

•

Our model is competitive with state-of-the-art methods on a public dataset.

Background and purpose

Convolutional neural networks (CNNs) are increasingly used to automate segmentation for radiotherapy planning, where accurate segmentation of organs-at-risk (OARs) is crucial. Training CNNs often requires large amounts of data. However, large, high quality datasets are scarce. The aim of this study was to develop a CNN capable of accurate head and neck (HN) 3D auto-segmentation of planning CT scans using a small training dataset (34 CTs).

Materials and Method

Elements of our custom CNN architecture were varied to optimise segmentation performance. We tested and evaluated the impact of: using multiple contrast channels for the CT scan input at specific soft tissue and bony anatomy windows, resize vs. transpose convolutions, and loss functions based on overlap metrics and cross-entropy in different combinations. Model segmentation performance was compared with the inter-observer deviation of two doctors’ gold standard segmentations using the 95th percentile Hausdorff distance and mean distance-to-agreement (mDTA). The best performing configuration was further validated on a popular public dataset to compare with state-of-the-art (SOTA) auto-segmentation methods.

Results

Our best performing CNN configuration was competitive with current SOTA methods when evaluated on the public dataset with mDTA of (0.81±0.31) mm for the brainstem, (0.20±0.08) mm for the mandible, (0.77±0.14) mm for the left parotid and (0.81±0.28) mm for the right parotid.

Conclusions

Through careful tuning and customisation we trained a 3D CNN with a small dataset to produce segmentations of HN OARs with an accuracy that is comparable with inter-clinician deviations. Our proposed model performed competitively with current SOTA methods.

ThoraxNet: a 3D U-Net based two-stage framework for OAR segmentation on thoracic CT images

An important phase of radiation treatment planning is the accurate contouring of the organs at risk (OAR), which is necessary for the dose distribution calculation. The manual contouring approach currently used in clinical practice is tedious, time-consuming, and prone to inter and intra-observer variation. Therefore, a deep learning-based auto contouring tool can solve these issues by accurately delineating OARs on the computed tomography (CT) images. This paper proposes a two-stage deep learning-based segmentation model with an attention mechanism that automatically delineates OARs in thoracic CT images. After preprocessing the input CT volume, a 3D U-Net architecture will locate each organ to generate cropped images for the segmentation network. Next, two differently configured U-Net-based networks will perform the segmentation of large organs-left lung, right lung, heart, and small organs-esophagus and spinal cord, respectively. A post-processing step integrates all the individually-segmented organs to generate the final result. The suggested model outperformed the state-of-the-art approaches in terms of dice similarity coefficient (DSC) values for the lungs and the heart. It is worth mentioning that the proposed model achieved a dice score of 0.941, which is 1.1% higher than the best previous dice score, in the case of the heart, an important organ in the human body. Moreover, the clinical acceptance of the results is verified using dosimetric analysis. To delineate all five organs on a CT scan of size [Formula: see text], our model takes only 8.61 s. The proposed open-source automatic contouring tool can generate accurate contours in minimal time, consequently speeding up the treatment time and reducing the treatment cost.

A 3D segmentation network of mandible from CT scan with combination of multiple convolutional modules and edge supervision in mandibular reconstruction

Mandibular reconstruction is a very complex surgery that demands removing the tumor, which is followed by reconstruction of the defective mandible. Accurate segmentation of the mandible plays an important role in its preoperative planning. However, there are many segmentation challenges including the connected boundaries of upper and lower teeth, blurred condyle edges, metal artifact interference, and different shapes of the mandibles with tumor invasion (MTI). Those manual or semi-automatic segmentation methods commonly used in clinical practice are time-consuming and have poor effects. The automatic segmentation methods are mainly developed for the mandible without tumor invasion (Non-MTI) rather than MTI and have problems such as under-segmentation. Given these problems, this paper proposed a 3D automatic segmentation network of the mandible with a combination of multiple convolutional modules and edge supervision. Firstly, the squeeze-and-excitation residual module is used for feature optimization to make the network focused more on the mandibular segmentation region. Secondly, the multi atrous convolution cascade module is adapted to implement a multi-scale feature search to extract more detailed features. Considering that most mandibular segmentation networks ignore the boundary information, the loss function combining region loss and edge loss is applied to further improve the segmentation performance. The final experiment shows that the proposed network can segment Non-MTI and MTI quickly and automatically with an average segmentation time of 7.41s for a CT scan. In the meantime, it also has a good segmentation accuracy. For Non-MTI segmentation, the dice coefficient (Dice) reaches 97.98 ± 0.36%, average surface distance (ASD) reaches 0.061 ± 0.016 mm, and 95% Hausdorff distance (95HD) reaches 0.484 ± 0.027 mm. For Non-MTI segmentation, the Dice reaches 96.90 ± 1.59%, ASD reaches 0.162 ± 0.107 mm, and 95HD reaches 1.161 ± 1.034 mm. Compared with other methods, the proposed method has better segmentation performance, effectively improving segmentation accuracy and reducing under-segmentation. It can greatly improve doctor's segmentation efficiency and will have a promising application prospect in mandibular reconstruction surgery in the future.

Automatic mandible segmentation from CT image using 3D fully convolutional neural network based on DenseASPP and attention gates

PURPOSE

In cranio-maxillofacial surgery, it is of great clinical significance to segment mandible accurately and automatically from CT images. However, the connected region and blurred boundary in teeth and condyles make the process challenging. At present, the mandible is commonly segmented by experienced doctors using manually or semi-automatic methods, which is time-consuming and has poor segmentation consistency. In addition, existing automatic segmentation methods still have problems such as region misjudgment, low accuracy, and time-consuming.

METHODS

For these issues, an automatic mandibular segmentation method using 3d fully convolutional neural network based on densely connected atrous spatial pyramid pooling (DenseASPP) and attention gates (AG) was proposed in this paper. Firstly, the DenseASPP module was added to the network for extracting dense features at multiple scales. Thereafter, the AG module was applied in each skip connection to diminish irrelevant background information and make the network focus on segmentation regions. Finally, a loss function combining dice coefficient and focal loss was used to solve the imbalance among sample categories.

RESULTS

Test results showed that the proposed network obtained a relatively good segmentation result, with a Dice score of 97.588 ± 0.425%, Intersection over Union of 95.293 ± 0.812%, sensitivity of 96.252 ± 1.106%, average surface distance of 0.065 ± 0.020 mm and 95% Hausdorff distance of 0.491 ± 0.021 mm in segmentation accuracy. The comparison with other segmentation networks showed that our network not only had a relatively high segmentation accuracy but also effectively reduced the network's misjudgment. Meantime, the surface distance error also showed that our segmentation results were relatively close to the ground truth.

CONCLUSION

The proposed network has better segmentation performance and realizes accurate and automatic segmentation of the mandible. Furthermore, its segmentation time is 50.43 s for one CT scan, which greatly improves the doctor's work efficiency. It will have practical significance in cranio-maxillofacial surgery in the future.

Clinically Applicable Segmentation of Head and Neck Anatomy for Radiotherapy: Deep Learning Algorithm Development and Validation Study

BACKGROUND

Over half a million individuals are diagnosed with head and neck cancer each year globally. Radiotherapy is an important curative treatment for this disease, but it requires manual time to delineate radiosensitive organs at risk. This planning process can delay treatment while also introducing interoperator variability, resulting in downstream radiation dose differences. Although auto-segmentation algorithms offer a potentially time-saving solution, the challenges in defining, quantifying, and achieving expert performance remain.

OBJECTIVE

Adopting a deep learning approach, we aim to demonstrate a 3D U-Net architecture that achieves expert-level performance in delineating 21 distinct head and neck organs at risk commonly segmented in clinical practice.

METHODS

The model was trained on a data set of 663 deidentified computed tomography scans acquired in routine clinical practice and with both segmentations taken from clinical practice and segmentations created by experienced radiographers as part of this research, all in accordance with consensus organ at risk definitions.

RESULTS

We demonstrated the model's clinical applicability by assessing its performance on a test set of 21 computed tomography scans from clinical practice, each with 21 organs at risk segmented by 2 independent experts. We also introduced surface Dice similarity coefficient, a new metric for the comparison of organ delineation, to quantify the deviation between organ at risk surface contours rather than volumes, better reflecting the clinical task of correcting errors in automated organ segmentations. The model's generalizability was then demonstrated on 2 distinct open-source data sets, reflecting different centers and countries to model training.

CONCLUSIONS

Deep learning is an effective and clinically applicable technique for the segmentation of the head and neck anatomy for radiotherapy. With appropriate validation studies and regulatory approvals, this system could improve the efficiency, consistency, and safety of radiotherapy pathways.

Automatic Segmentation of Mandible from Conventional Methods to Deep Learning-A Review

Medical imaging techniques, such as (cone beam) computed tomography and magnetic resonance imaging, have proven to be a valuable component for oral and maxillofacial surgery (OMFS). Accurate segmentation of the mandible from head and neck (H&N) scans is an important step in order to build a personalized 3D digital mandible model for 3D printing and treatment planning of OMFS. Segmented mandible structures are used to effectively visualize the mandible volumes and to evaluate particular mandible properties quantitatively. However, mandible segmentation is always challenging for both clinicians and researchers, due to complex structures and higher attenuation materials, such as teeth (filling) or metal implants that easily lead to high noise and strong artifacts during scanning. Moreover, the size and shape of the mandible vary to a large extent between individuals. Therefore, mandible segmentation is a tedious and time-consuming task and requires adequate training to be performed properly. With the advancement of computer vision approaches, researchers have developed several algorithms to automatically segment the mandible during the last two decades. The objective of this review was to present the available fully (semi)automatic segmentation methods of the mandible published in different scientific articles. This review provides a vivid description of the scientific advancements to clinicians and researchers in this field to help develop novel automatic methods for clinical applications.

A Preliminary Experience of Implementing Deep-Learning Based Auto-Segmentation in Head and Neck Cancer: A Study on Real-World Clinical Cases

Purpose

While artificial intelligence has shown great promise in organs-at-risk (OARs) auto segmentation for head and neck cancer (HNC) radiotherapy, to reach the level of clinical acceptance of this technology in real-world routine practice is still a challenge. The purpose of this study was to validate a U-net-based full convolutional neural network (CNN) for the automatic delineation of OARs of HNC, focusing on clinical implementation and evaluation.

Methods

In the first phase, the CNN was trained on 364 clinical HNC patients’ CT images with annotated contouring from routine clinical cases by different oncologists. The automated delineation accuracy was quantified using the Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD). To assess efficiency, the time required to edit the auto-contours to a clinically acceptable standard was evaluated by a questionnaire. For subjective evaluation, expert oncologists (more than 10 years’ experience) were randomly presented with automated delineations or manual contours of 15 OARs for 30 patient cases. In the second phase, the network was retrained with an additional 300 patients, which were generated by pre-trained CNN and edited by oncologists until to meet clinical acceptance.

Results

Based on DSC, the CNN performed best for the spinal cord, brainstem, temporal lobe, eyes, optic nerve, parotid glands and larynx (DSC >0.7). Higher conformity for the OARs delineation was achieved by retraining our architecture, largest DSC improvement on oral cavity (0.53 to 0.93). Compared with the manual delineation time, after using auto-contouring, this duration was significantly shortened from hours to minutes. In the subjective evaluation, two observes showed an apparent inclination on automatic OARs contouring, even for relatively low DSC values. Most of the automated OARs segmentation can reach the clinical acceptance level compared to manual delineations.

Conclusions

After retraining, the CNN developed for OARs automated delineation in HNC was proved to be more robust, efficiency and consistency in clinical practice. Deep learning-based auto-segmentation shows great potential to alleviate the labor-intensive contouring of OAR for radiotherapy treatment planning.

A review of deep learning based methods for medical image multi-organ segmentation

Deep learning has revolutionized image processing and achieved the-state-of-art performance in many medical image segmentation tasks. Many deep learning-based methods have been published to segment different parts of the body for different medical applications. It is necessary to summarize the current state of development for deep learning in the field of medical image segmentation. In this paper, we aim to provide a comprehensive review with a focus on multi-organ image segmentation, which is crucial for radiotherapy where the tumor and organs-at-risk need to be contoured for treatment planning. We grouped the surveyed methods into two broad categories which are 'pixel-wise classification' and 'end-to-end segmentation'. Each category was divided into subgroups according to their network design. For each type, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review, we discussed the achievements, shortcomings and future potentials of each category. To enable direct comparison, we listed the performance of the surveyed works that used thoracic and head-and-neck benchmark datasets.

Artificial intelligence and machine learning for medical imaging: A technology review

Artificial intelligence (AI) has recently become a very popular buzzword, as a consequence of disruptive technical advances and impressive experimental results, notably in the field of image analysis and processing. In medicine, specialties where images are central, like radiology, pathology or oncology, have seized the opportunity and considerable efforts in research and development have been deployed to transfer the potential of AI to clinical applications. With AI becoming a more mainstream tool for typical medical imaging analysis tasks, such as diagnosis, segmentation, or classification, the key for a safe and efficient use of clinical AI applications relies, in part, on informed practitioners. The aim of this review is to present the basic technological pillars of AI, together with the state-of-the-art machine learning methods and their application to medical imaging. In addition, we discuss the new trends and future research directions. This will help the reader to understand how AI methods are now becoming an ubiquitous tool in any medical image analysis workflow and pave the way for the clinical implementation of AI-based solutions.

Automatic detection and segmentation of multiple brain metastases on magnetic resonance image using asymmetric UNet architecture

Detection of brain metastases is a paramount task in cancer management due both to the number of high-risk patients and the difficulty of achieving consistent detection. In this study, we aim to improve the accuracy of automated brain metastasis (BM) detection methods using a novel asymmetric UNet (asym-UNet) architecture. An end-to-end asymmetric 3D-UNet architecture, with two down-sampling arms and one up-sampling arm, was constructed to capture the imaging features. The two down-sampling arms were trained using two different kernels (3 × 3 × 3 and 1 × 1 × 3, respectively) with the kernel (1 × 1 × 3) dominating the learning. As a comparison, vanilla single 3D UNets were trained with different kernels and evaluated using the same datasets. Voxel-based Dice similarity coefficient (DSC_v), sensitivity (S _v), precision (P _v), BM-based sensitivity (S _BM), and false detection rate (F _BM) were used to evaluate model performance. Contrast-enhanced T1 MR images from 195 patients with a total of 1034 BMs were solicited from our institutional stereotactic radiosurgery database. The patient cohort was split into training (160 patients, 809 lesions), validation (20 patients, 136 lesions), and testing (15 patients, 89 lesions) datasets. The lesions in the testing dataset were further divided into two subgroups based on the diameters (small S = 1-10 mm, large L = 11-26 mm). In the testing dataset, there were 72 and 17 BMs in the S and L sub-groups, respectively. Among all trained networks, asym-UNet achieved the highest DSC_v of 0.84 and lowest F _BM of 0.24. Although vanilla 3D-UNet with a single 1 × 1 × 3 kernel achieved the highest sensitivities for the S group, it resulted in the lowest precision and highest false detection rate. Asym-UNet was shown to balance sensitivity and false detection rate as well as keep the segmentation accuracy high. The novel asym-UNet segmentation network showed overall competitive segmentation performance and more pronounced improvement in hard-to-detect small BMs comparing to the vanilla single 3D UNet.

Auto-segmentation of organs at risk for head and neck radiotherapy planning: From atlas-based to deep learning methods

Radiotherapy (RT) is one of the basic treatment modalities for cancer of the head and neck (H&N), which requires a precise spatial description of the target volumes and organs at risk (OARs) to deliver a highly conformal radiation dose to the tumor cells while sparing the healthy tissues. For this purpose, target volumes and OARs have to be delineated and segmented from medical images. As manual delineation is a tedious and time-consuming task subjected to intra/interobserver variability, computerized auto-segmentation has been developed as an alternative. The field of medical imaging and RT planning has experienced an increased interest in the past decade, with new emerging trends that shifted the field of H&N OAR auto-segmentation from atlas-based to deep learning-based approaches. In this review, we systematically analyzed 78 relevant publications on auto-segmentation of OARs in the H&N region from 2008 to date, and provided critical discussions and recommendations from various perspectives: image modality - both computed tomography and magnetic resonance image modalities are being exploited, but the potential of the latter should be explored more in the future; OAR - the spinal cord, brainstem, and major salivary glands are the most studied OARs, but additional experiments should be conducted for several less studied soft tissue structures; image database - several image databases with the corresponding ground truth are currently available for methodology evaluation, but should be augmented with data from multiple observers and multiple institutions; methodology - current methods have shifted from atlas-based to deep learning auto-segmentation, which is expected to become even more sophisticated; ground truth - delineation guidelines should be followed and participation of multiple experts from multiple institutions is recommended; performance metrics - the Dice coefficient as the standard volumetric overlap metrics should be accompanied with at least one distance metrics, and combined with clinical acceptability scores and risk assessments; segmentation performance - the best performing methods achieve clinically acceptable auto-segmentation for several OARs, however, the dosimetric impact should be also studied to provide clinically relevant endpoints for RT planning.