Segmentation of Pathological Structures by Landmark-Assisted Deformable Models

SpineMamba: Enhancing 3D spinal segmentation in clinical imaging through residual visual Mamba layers and shape priors

Accurate segmentation of three-dimensional (3D) clinical medical images is critical for the diagnosis and treatment of spinal diseases. However, the complexity of spinal anatomy and the inherent uncertainties of current imaging technologies pose significant challenges for the semantic segmentation of spinal images. Although convolutional neural networks (CNNs) and Transformer-based models have achieved remarkable progress in spinal segmentation, their limitations in modeling long-range dependencies hinder further improvements in segmentation accuracy. To address these challenges, we propose a novel framework, SpineMamba, which incorporates a residual visual Mamba layer capable of effectively capturing and modeling the deep semantic features and long-range spatial dependencies in 3D spinal data. To further enhance the structural semantic understanding of the vertebrae, we also propose a novel spinal shape prior module that captures specific anatomical information about the spine from medical images, significantly enhancing the model's ability to extract structural semantic information of the vertebrae. Extensive comparative and ablation experiments across three datasets demonstrate that SpineMamba outperforms existing state-of-the-art models. On two computed tomography (CT) datasets, the average Dice similarity coefficients achieved are 94.40±4% and 88.28±3%, respectively, while on a magnetic resonance (MR) dataset, the model achieves a Dice score of 86.95±10%. Notably, SpineMamba surpasses the widely recognized nnU-Net in segmentation accuracy, with a maximum improvement of 3.63 percentage points. These results highlight the precision, robustness, and exceptional generalization capability of SpineMamba.

A spatio-temporal graph convolutional network for ultrasound echocardiographic landmark detection

Landmark detection is a crucial task in medical image analysis, with applications across various fields. However, current methods struggle to accurately locate landmarks in medical images with blurred tissue boundaries due to low image quality. In particular, in echocardiography, sparse annotations make it challenging to predict landmarks with position stability and temporal consistency. In this paper, we propose a spatio-temporal graph convolutional network tailored for echocardiography landmark detection. We specifically sample landmark labels from the left ventricular endocardium and pre-calculate their correlations to establish structural priors. Our approach involves a graph convolutional neural network that learns the interrelationships among landmarks, significantly enhancing landmark accuracy within ambiguous tissue contexts. Additionally, we integrate gate recurrent units to grasp the temporal consistency of landmarks across consecutive images, augmenting the model's resilience against unlabeled data. Through validation across three echocardiography datasets, our method demonstrates superior accuracy when contrasted with alternative landmark detection models.

Better Rough Than Scarce: Proximal Femur Fracture Segmentation With Rough Annotations

Proximal femoral fracture segmentation in computed tomography (CT) is essential in the preoperative planning of orthopedic surgeons. Recently, numerous deep learning-based approaches have been proposed for segmenting various structures within CT scans. Nevertheless, distinguishing various attributes between fracture fragments and soft tissue regions in CT scans frequently poses challenges, which have received comparatively limited research attention. Besides, the cornerstone of contemporary deep learning methodologies is the availability of annotated data, while detailed CT annotations remain scarce. To address the challenge, we propose a novel weakly-supervised framework, namely Rough Turbo Net (RT-Net), for the segmentation of proximal femoral fractures. We emphasize the utilization of human resources to produce rough annotations on a substantial scale, as opposed to relying on limited fine-grained annotations that demand a substantial time to create. In RT-Net, rough annotations pose fractured-region constraints, which have demonstrated significant efficacy in enhancing the accuracy of the network. Conversely, the fine annotations can provide more details for recognizing edges and soft tissues. Besides, we design a spatial adaptive attention module (SAAM) that adapts to the spatial distribution of the fracture regions and align feature in each decoder. Moreover, we propose a fine-edge loss which is applied through an edge discrimination network to penalize the absence or imprecision edge features. Extensive quantitative and qualitative experiments demonstrate the superiority of RT-Net to state-of-the-art approaches. Furthermore, additional experiments show that RT-Net has the capability to produce pseudo labels for raw CT images that can further improve fracture segmentation performance and has the potential to improve segmentation performance on public datasets. The code is available at: https://github.com/zyairelu/RT-Net.

The use of deep learning in medical imaging to improve spine care: A scoping review of current literature and clinical applications

Background

Artificial intelligence is a revolutionary technology that promises to assist clinicians in improving patient care. In radiology, deep learning (DL) is widely used in clinical decision aids due to its ability to analyze complex patterns and images. It allows for rapid, enhanced data, and imaging analysis, from diagnosis to outcome prediction. The purpose of this study was to evaluate the current literature and clinical utilization of DL in spine imaging.

Methods

This study is a scoping review and utilized the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology to review the scientific literature from 2012 to 2021. A search in PubMed, Web of Science, Embased, and IEEE Xplore databases with syntax specific for DL and medical imaging in spine care applications was conducted to collect all original publications on the subject. Specific data was extracted from the available literature, including algorithm application, algorithms tested, database type and size, algorithm training method, and outcome of interest.

Results

A total of 365 studies (total sample of 232,394 patients) were included and grouped into 4 general applications: diagnostic tools, clinical decision support tools, automated clinical/instrumentation assessment, and clinical outcome prediction. Notable disparities exist in the selected algorithms and the training across multiple disparate databases. The most frequently used algorithms were U-Net and ResNet. A DL model was developed and validated in 92% of included studies, while a pre-existing DL model was investigated in 8%. Of all developed models, only 15% of them have been externally validated.

Conclusions

Based on this scoping review, DL in spine imaging is used in a broad range of clinical applications, particularly for diagnosing spinal conditions. There is a wide variety of DL algorithms, database characteristics, and training methods. Future studies should focus on external validation of existing models before bringing them into clinical use.

An overview of artificial intelligence in medical physics and radiation oncology

Artificial intelligence (AI) is developing rapidly and has found widespread applications in medicine, especially radiotherapy. This paper provides a brief overview of AI applications in radiotherapy, and highlights the research directions of AI that can potentially make significant impacts and relevant ongoing research works in these directions. Challenging issues related to the clinical applications of AI, such as robustness and interpretability of AI models, are also discussed. The future research directions of AI in the field of medical physics and radiotherapy are highlighted.

Vertebrae localization, segmentation and identification using a graph optimization and an anatomic consistency cycle

Vertebrae localization, segmentation and identification in CT images is key to numerous clinical applications. While deep learning strategies have brought to this field significant improvements over recent years, transitional and pathological vertebrae are still plaguing most existing approaches as a consequence of their poor representation in training datasets. Alternatively, proposed non-learning based methods take benefit of prior knowledge to handle such particular cases. In this work we propose to combine both strategies. To this purpose we introduce an iterative cycle in which individual vertebrae are recurrently localized, segmented and identified using deep-networks, while anatomic consistency is enforced using statistical priors. In this strategy, the transitional vertebrae identification is handled by encoding their configurations in a graphical model that aggregates local deep-network predictions into an anatomically consistent final result. Our approach achieves the state-of-the-art results on the VerSe20 challenge benchmark, and outperforms all methods on transitional vertebrae as well as the generalization to the VerSe19 challenge benchmark. Furthermore, our method can detect and report inconsistent spine regions that do not satisfy the anatomic consistency priors. Our code and model are openly available for research purposes.¹.

CT-Based Automatic Spine Segmentation Using Patch-Based Deep Learning

CT vertebral segmentation plays an essential role in various clinical applications, such as computer-assisted surgical interventions, assessment of spinal abnormalities, and vertebral compression fractures. Automatic CT vertebral segmentation is challenging due to the overlapping shadows of thoracoabdominal structures such as the lungs, bony structures such as the ribs, and other issues such as ambiguous object borders, complicated spine architecture, patient variability, and fluctuations in image contrast. Deep learning is an emerging technique for disease diagnosis in the medical field. This study proposes a patch-based deep learning approach to extract the discriminative features from unlabeled data using a stacked sparse autoencoder (SSAE). 2D slices from a CT volume are divided into overlapping patches fed into the model for training. A random under sampling (RUS)-module is applied to balance the training data by selecting a subset of the majority class. SSAE uses pixel intensities alone to learn high-level features to recognize distinctive features from image patches. Each image is subjected to a sliding window operation to express image patches using autoencoder high-level features, which are then fed into a sigmoid layer to classify whether each patch is a vertebra or not. We validate our approach on three diverse publicly available datasets: VerSe, CSI-Seg, and the Lumbar CT dataset. Our proposed method outperformed other models after configuration optimization by achieving 89.9% in precision, 90.2% in recall, 98.9% in accuracy, 90.4% in F-score, 82.6% in intersection over union (IoU), and 90.2% in Dice coefficient (DC). The results of this study demonstrate that our model’s performance consistency using a variety of validation strategies is flexible, fast, and generalizable, making it suited for clinical application.

A novel 3D lumbar vertebrae location and segmentation method based on the fusion envelope of 2D hybrid visual projection images

In recent years, fast and precise lumbar vertebrae segmentation technology have been one of the important topics in practical medical diagnosis and assisted medical surgery scenarios. However, most of the existing vertebral segmentation methods are based on the whole vertebral scanning space, which, up to some extent, is difficult to meet the clinical needs because of its large time complexity and space complexity. Different from the existing methods, for better exploiting the real time of lumbar segmentation, meanwhile ensuring its accuracy, a novel 3D lumbar vertebrae location and segmentation method based on the fusion envelope of 2D hybrid visual projection images (LVLS-HVPFE) is proposed in this paper. Firstly, a 2D projection location network of lumbar vertebrae based on fusion envelope of hybrid visual projection images is proposed to obtain the accurate location of each intact lumbar vertebra in the coronal and sagittal planes respectively. Among them, the envelope dataset of hybrid visual projection images (ED^HVPs) is established to enhance feature representation and suppress interference in the process of dimensionality reduction projection. An envelope deep neural network (EDNN) for ED^HVPs is established to effectively obtain depth envelope structure features with three different sizes, and a dimension reduction fusion mechanism is proposed to increase the sampling density of features and ensure the mutual independence of multi-scale features. Secondly, the concept of 3D localization criterion with spatial dimensionality reduction (SDRLC) is first proposed as a measure to verify the distribution consistency of vertebral targets in coronal and sagittal planes of a CT scan, and it can directionally guide for the subsequent 3D lumbar segmentation. Thirdly, under the condition of 3D positioning subspace of each intact lumbar vertebra, the 3D segmentation network based on spatial orientation guidance is used to realize an accurate segmentation of corresponding lumbar vertebra. The proposed method is evaluated with three representative datasets, and experimental results show that it is superior to the state-of-the-art methods.

A deep learning framework for vertebral morphometry and Cobb angle measurement with external validation

PURPOSE

To propose a fully automated deep learning (DL) framework for the vertebral morphometry and Cobb angle measurement from three-dimensional (3D) computed tomography (CT) images of the spine, and validate the proposed framework on an external database.

METHODS

The vertebrae were first localized and segmented in each 3D CT image using a DL architecture based on an ensemble of U-Nets, and then automated vertebral morphometry in the form of vertebral body (VB) and intervertebral disk (IVD) heights, and spinal curvature measurements in the form of coronal and sagittal Cobb angles (thoracic kyphosis and lumbar lordosis) were performed using dedicated machine learning techniques. The framework was trained on 1725 vertebrae from 160 CT images and validated on an external database of 157 vertebrae from 15 CT images.

RESULTS

The resulting mean absolute errors (± standard deviation) between the obtained DL and corresponding manual measurements were 1.17 ± 0.40 mm for VB heights, 0.54 ± 0.21 mm for IVD heights, and 3.42 ± 1.36° for coronal and sagittal Cobb angles, with respective maximal absolute errors of 2.51 mm, 1.64 mm, and 5.52°. Linear regression revealed excellent agreement, with Pearson's correlation coefficient of 0.943, 0.928, and 0.996, respectively.

CONCLUSION

The obtained results are within the range of values, obtained by existing DL approaches without external validation. The results therefore confirm the scalability of the proposed DL framework from the perspective of application to external data, and time and computational resource consumption required for framework training.

Learning to Localize Cross-Anatomy Landmarks in X-Ray Images with a Universal Model

Objective and Impact Statement. In this work, we develop a universal anatomical landmark detection model which learns once from multiple datasets corresponding to different anatomical regions. Compared with the conventional model trained on a single dataset, this universal model not only is more light weighted and easier to train but also improves the accuracy of the anatomical landmark location. Introduction. The accurate and automatic localization of anatomical landmarks plays an essential role in medical image analysis. However, recent deep learning-based methods only utilize limited data from a single dataset. It is promising and desirable to build a model learned from different regions which harnesses the power of big data. Methods. Our model consists of a local network and a global network, which capture local features and global features, respectively. The local network is a fully convolutional network built up with depth-wise separable convolutions, and the global network uses dilated convolution to enlarge the receptive field to model global dependencies. Results. We evaluate our model on four 2D X-ray image datasets totaling 1710 images and 72 landmarks in four anatomical regions. Extensive experimental results show that our model improves the detection accuracy compared to the state-of-the-art methods. Conclusion. Our model makes the first attempt to train a single network on multiple datasets for landmark detection. Experimental results qualitatively and quantitatively show that our proposed model performs better than other models trained on multiple datasets and even better than models trained on a single dataset separately.

Automated segmentation of the fractured vertebrae on CT and its applicability in a radiomics model to predict fracture malignancy

Although CT radiomics has shown promising results in the evaluation of vertebral fractures, the need for manual segmentation of fractured vertebrae limited the routine clinical implementation of radiomics. Therefore, automated segmentation of fractured vertebrae is needed for successful clinical use of radiomics. In this study, we aimed to develop and validate an automated algorithm for segmentation of fractured vertebral bodies on CT, and to evaluate the applicability of the algorithm in a radiomics prediction model to differentiate benign and malignant fractures. A convolutional neural network was trained to perform automated segmentation of fractured vertebral bodies using 341 vertebrae with benign or malignant fractures from 158 patients, and was validated on independent test sets (internal test, 86 vertebrae [59 patients]; external test, 102 vertebrae [59 patients]). Then, a radiomics model predicting fracture malignancy on CT was constructed, and the prediction performance was compared between automated and human expert segmentations. The algorithm achieved good agreement with human expert segmentation at testing (Dice similarity coefficient, 0.93-0.94; cross-sectional area error, 2.66-2.97%; average surface distance, 0.40-0.54 mm). The radiomics model demonstrated good performance in the training set (AUC, 0.93). In the test sets, automated and human expert segmentations showed comparable prediction performances (AUC, internal test, 0.80 vs 0.87, p = 0.044; external test, 0.83 vs 0.80, p = 0.37). In summary, we developed and validated an automated segmentation algorithm that showed comparable performance to human expert segmentation in a CT radiomics model to predict fracture malignancy, which may enable more practical clinical utilization of radiomics.

Artificial intelligence in image-guided radiotherapy: a review of treatment target localization

Modern conformal beam delivery techniques require image-guidance to ensure the prescribed dose to be delivered as planned. Recent advances in artificial intelligence (AI) have greatly augmented our ability to accurately localize the treatment target while sparing the normal tissues. In this paper, we review the applications of AI-based algorithms in image-guided radiotherapy (IGRT), and discuss the indications of these applications to the future of clinical practice of radiotherapy. The benefits, limitations and some important trends in research and development of the AI-based IGRT techniques are also discussed. AI-based IGRT techniques have the potential to monitor tumor motion, reduce treatment uncertainty and improve treatment precision. Particularly, these techniques also allow more healthy tissue to be spared while keeping tumor coverage the same or even better.

Artificial Intelligence and Computer Vision in Low Back Pain: A Systematic Review

Chronic Low Back Pain (LBP) is a symptom that may be caused by several diseases, and it is currently the leading cause of disability worldwide. The increased amount of digital images in orthopaedics has led to the development of methods related to artificial intelligence, and to computer vision in particular, which aim to improve diagnosis and treatment of LBP. In this manuscript, we have systematically reviewed the available literature on the use of computer vision in the diagnosis and treatment of LBP. A systematic research of PubMed electronic database was performed. The search strategy was set as the combinations of the following keywords: "Artificial Intelligence", "Feature Extraction", "Segmentation", "Computer Vision", "Machine Learning", "Deep Learning", "Neural Network", "Low Back Pain", "Lumbar". Results: The search returned a total of 558 articles. After careful evaluation of the abstracts, 358 were excluded, whereas 124 papers were excluded after full-text examination, taking the number of eligible articles to 76. The main applications of computer vision in LBP include feature extraction and segmentation, which are usually followed by further tasks. Most recent methods use deep learning models rather than digital image processing techniques. The best performing methods for segmentation of vertebrae, intervertebral discs, spinal canal and lumbar muscles achieve Sørensen-Dice scores greater than 90%, whereas studies focusing on localization and identification of structures collectively showed an accuracy greater than 80%. Future advances in artificial intelligence are expected to increase systems' autonomy and reliability, thus providing even more effective tools for the diagnosis and treatment of LBP.

Precise laminae segmentation based on neural network for robot-assisted decompressive laminectomy

BACKGROUND AND OBJECTIVE

The decompressive laminectomy is one of the most common operations to treat lumbar spinal stenosis by removing the laminae above the spinal nerve. Recently, an increasing number of robots are deployed during the surgical process to reduce the burden on surgeons and to reduce complications. However, for the robot-assisted decompressive laminectomy, an accurate 3D model of laminae from a CT image is highly desired. The purpose of this paper is to precisely segment the laminae with fewer calculations.

METHODS

We propose a two-stage neural network SegRe-Net. In the first stage, the entire intraoperative CT image is inputted to acquire the coarse segmentation of vertebrae with low resolution and the probability map of the laminar centers. The second stage is trained to refine the segmentation of laminae.

RESULTS

Three public available datasets were used to train and validate the models. The experimental results demonstrated the effectiveness of the proposed network on laminar segmentation with an average Dice coefficient of 96.38% and an average symmetric surface distance of 0.097 mm.

CONCLUSION

The proposed two-stage network can achieve better results than those baseline models in the laminae segmentation task with less calculation amount and learnable parameters. Our methods improve the accuracy of laminar models and reduce the image processing time. It can be used to provide a more precise planning trajectory and may promote the clinical application for the robot-assisted decompression laminectomy surgery.

A Region-Based Deep Level Set Formulation for Vertebral Bone Segmentation of Osteoporotic Fractures

Accurate segmentation of the vertebrae from medical images plays an important role in computer-aided diagnoses (CADs). It provides an initial and early diagnosis of various vertebral abnormalities to doctors and radiologists. Vertebrae segmentation is very important but difficult task in medical imaging due to low-contrast imaging and noise. It becomes more challenging when dealing with fractured (osteoporotic) cases. This work is dedicated to address the challenging problem of vertebra segmentation. In the past, various segmentation techniques of vertebrae have been proposed. Recently, deep learning techniques have been introduced in biomedical image processing for segmentation and characterization of several abnormalities. These techniques are becoming popular for segmentation purposes due to their robustness and accuracy. In this paper, we present a novel combination of traditional region-based level set with deep learning framework in order to predict shape of vertebral bones accurately; thus, it would be able to handle the fractured cases efficiently. We termed this novel Framework as "FU-Net" which is a powerful and practical framework to handle fractured vertebrae segmentation efficiently. The proposed method was successfully evaluated on two different challenging datasets: (1) 20 CT scans, 15 healthy cases, and 5 fractured cases provided at spine segmentation challenge CSI 2014; (2) 25 CT image data (both healthy and fractured cases) provided at spine segmentation challenge CSI 2016 or xVertSeg.v1 challenge. We have achieved promising results on our proposed technique especially on fractured cases. Dice score was found to be 96.4 ± 0.8% without fractured cases and 92.8 ± 1.9% with fractured cases in CSI 2014 dataset (lumber and thoracic). Similarly, dice score was 95.2 ± 1.9% on 15 CT dataset (with given ground truths) and 95.4 ± 2.1% on total 25 CT dataset for CSI 2016 datasets (with 10 annotated CT datasets). The proposed technique outperformed other state-of-the-art techniques and handled the fractured cases for the first time efficiently.

VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images

Vertebral labelling and segmentation are two fundamental tasks in an automated spine processing pipeline. Reliable and accurate processing of spine images is expected to benefit clinical decision support systems for diagnosis, surgery planning, and population-based analysis of spine and bone health. However, designing automated algorithms for spine processing is challenging predominantly due to considerable variations in anatomy and acquisition protocols and due to a severe shortage of publicly available data. Addressing these limitations, the Large Scale Vertebrae Segmentation Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020, with a call for algorithms tackling the labelling and segmentation of vertebrae. Two datasets containing a total of 374 multi-detector CT scans from 355 patients were prepared and 4505 vertebrae have individually been annotated at voxel level by a human-machine hybrid algorithm (https://osf.io/nqjyw/, https://osf.io/t98fz/). A total of 25 algorithms were benchmarked on these datasets. In this work, we present the results of this evaluation and further investigate the performance variation at the vertebra level, scan level, and different fields of view. We also evaluate the generalisability of the approaches to an implicit domain shift in data by evaluating the top-performing algorithms of one challenge iteration on data from the other iteration. The principal takeaway from VerSe: the performance of an algorithm in labelling and segmenting a spine scan hinges on its ability to correctly identify vertebrae in cases of rare anatomical variations. The VerSe content and code can be accessed at: https://github.com/anjany/verse.

Radiomic and radiogenomic modeling for radiotherapy: strategies, pitfalls, and challenges

The power of predictive modeling for radiotherapy outcomes has historically been limited by an inability to adequately capture patient-specific variabilities; however, next-generation platforms together with imaging technologies and powerful bioinformatic tools have facilitated strategies and provided optimism. Integrating clinical, biological, imaging, and treatment-specific data for more accurate prediction of tumor control probabilities or risk of radiation-induced side effects are high-dimensional problems whose solutions could have widespread benefits to a diverse patient population-we discuss technical approaches toward this objective. Increasing interest in the above is specifically reflected by the emergence of two nascent fields, which are distinct but complementary: radiogenomics, which broadly seeks to integrate biological risk factors together with treatment and diagnostic information to generate individualized patient risk profiles, and radiomics, which further leverages large-scale imaging correlates and extracted features for the same purpose. We review classical analytical and data-driven approaches for outcomes prediction that serve as antecedents to both radiomic and radiogenomic strategies. Discussion then focuses on uses of conventional and deep machine learning in radiomics. We further consider promising strategies for the harmonization of high-dimensional, heterogeneous multiomics datasets (panomics) and techniques for nonparametric validation of best-fit models. Strategies to overcome common pitfalls that are unique to data-intensive radiomics are also discussed.

Automatic detection and segmentation of lumbar vertebrae from X-ray images for compression fracture evaluation

For compression fracture detection and evaluation, an automatic X-ray image segmentation technique that combines deep-learning and level-set methods is proposed. Automatic segmentation is much more difficult for X-ray images than for CT or MRI images because they contain overlapping shadows of thoracoabdominal structures including lungs, bowel gases, and other bony structures such as ribs. Additional difficulties include unclear object boundaries, the complex shape of the vertebra, inter-patient variability, and variations in image contrast. Accordingly, a structured hierarchical segmentation method is presented that combines the advantages of two deep-learning methods. Pose-driven learning is used to selectively identify the five lumbar vertebrae in an accurate and robust manner. With knowledge of the vertebral positions, M-net is employed to segment the individual vertebra. Finally, fine-tuning segmentation is applied by combining the level-set method with the previously obtained segmentation results. The performance of the proposed method was validated by 160 lumbar X-ray images, resulting in a mean Dice similarity metric of 91.60±2.22%. The results show that the proposed method achieves accurate and robust identification of each lumbar vertebra and fine segmentation of individual vertebra.

Grinding trajectory generator in robot-assisted laminectomy surgery

PURPOSE

Grinding trajectory planning for robot-assisted laminectomy is a complicated and cumbersome task. The purpose of this research is to automatically obtain the surgical target area from the CT image, and based on this, formulate a reasonable robotic grinding trajectory.

METHODS

We propose a deep neural network for laminae positioning, a trajectory generation strategy, and a grinding speed adjusting strategy. These algorithms can obtain surgical information from CT images and automatically complete grinding trajectory planning.

RESULTS

The proposed laminae positioning network can reach a recognition accuracy of 95.7%, and the positioning error is only 1.12 mm in the desired direction. The simulated surgical planning on the public dataset has achieved the expected results. In a set of comparative robotic grinding experiments, those using the speed adjustment algorithm obtained a smoother grinding force.

CONCLUSION

Our work can automatically extract laminar centers from the CT image precisely to formulate a reasonable surgical trajectory plan. It simplifies the surgical planning process and reduces the time needed for surgeons to perform such a cumbersome operation manually.

Automated Pipeline to Generate Anatomically Accurate Patient-Specific Biomechanical Models of Healthy and Pathological FSUs

State-of-the-art preoperative biomechanical analysis for the planning of spinal surgery not only requires the generation of three-dimensional patient-specific models but also the accurate biomechanical representation of vertebral joints. The benefits offered by computational models suitable for such purposes are still outweighed by the time and effort required for their generation, thus compromising their applicability in a clinical environment. In this work, we aim to ease the integration of computerized methods into patient-specific planning of spinal surgery. We present the first pipeline combining deep learning and finite element methods that allows a completely automated model generation of functional spine units (FSUs) of the lumbar spine for patient-specific FE simulations (FEBio). The pipeline consists of three steps: (a) multiclass segmentation of cropped 3D CT images containing lumbar vertebrae using the DenseVNet network, (b) automatic landmark-based mesh fitting of statistical shape models onto 3D semantic segmented meshes of the vertebral models, and (c) automatic generation of patient-specific FE models of lumbar segments for the simulation of flexion-extension, lateral bending, and axial rotation movements. The automatic segmentation of FSUs was evaluated against the gold standard (manual segmentation) using 10-fold cross-validation. The obtained Dice coefficient was 93.7% on average, with a mean surface distance of 0.88 mm and a mean Hausdorff distance of 11.16 mm (N = 150). Automatic generation of finite element models to simulate the range of motion (ROM) was successfully performed for five healthy and five pathological FSUs. The results of the simulations were evaluated against the literature and showed comparable ROMs in both healthy and pathological cases, including the alteration of ROM typically observed in severely degenerated FSUs. The major intent of this work is to automate the creation of anatomically accurate patient-specific models by a single pipeline allowing functional modeling of spinal motion in healthy and pathological FSUs. Our approach reduces manual efforts to a minimum and the execution of the entire pipeline including simulations takes approximately 2 h. The automation, time-efficiency and robustness level of the pipeline represents a first step toward its clinical integration.

Fluid Lubrication and Cooling Effects in Diamond Grinding of Human Iliac Bone

Background and Objectives: Although there have been research on bone cutting, there have been few research on bone grinding. This study reports the measurement results of the experimental system that simulated partial laminectomy in microscopic spine surgery. The purpose of this study was to examine the fluid lubrication and cooling in bone grinding, histological characteristics of workpieces, and differences in grinding between manual and milling machines. Materials and Methods: Thiel-fixed human iliac bones were used as workpieces. A neurosurgical microdrill was used as a drill system. The workpieces were fixed to a 4-component piezo-electric dynamometer and fixtures, which was used to measure the triaxial power during bone grinding. Grinding tasks were performed by manual activity and a small milling machine with or without water. Results: In bone grinding with 4-mm diameter diamond burs and water, reduction in the number of sudden increases in grinding resistance and cooling effect of over 100 °C were confirmed. Conclusion: Manual grinding may enable the control of the grinding speed and cutting depth while giving top priority to uniform torque on the work piece applied by tools. Observing the drill tip using a triaxial dynamometer in the quantification of surgery may provide useful data for the development of safety mechanisms to prevent a sudden deviation of the drill tip.

SpineParseNet: Spine Parsing for Volumetric MR Image by a Two-Stage Segmentation Framework With Semantic Image Representation

Spine parsing (i.e., multi-class segmentation of vertebrae and intervertebral discs (IVDs)) for volumetric magnetic resonance (MR) image plays a significant role in various spinal disease diagnoses and treatments of spine disorders, yet is still a challenge due to the inter-class similarity and intra-class variation of spine images. Existing fully convolutional network based methods failed to explicitly exploit the dependencies between different spinal structures. In this article, we propose a novel two-stage framework named SpineParseNet to achieve automated spine parsing for volumetric MR images. The SpineParseNet consists of a 3D graph convolutional segmentation network (GCSN) for 3D coarse segmentation and a 2D residual U-Net (ResUNet) for 2D segmentation refinement. In 3D GCSN, region pooling is employed to project the image representation to graph representation, in which each node representation denotes a specific spinal structure. The adjacency matrix of the graph is designed according to the connection of spinal structures. The graph representation is evolved by graph convolutions. Subsequently, the proposed region unpooling module re-projects the evolved graph representation to a semantic image representation, which facilitates the 3D GCSN to generate reliable coarse segmentation. Finally, the 2D ResUNet refines the segmentation. Experiments on T2-weighted volumetric MR images of 215 subjects show that SpineParseNet achieves impressive performance with mean Dice similarity coefficients of 87.32 ± 4.75%, 87.78 ± 4.64%, and 87.49 ± 3.81% for the segmentations of 10 vertebrae, 9 IVDs, and all 19 spinal structures respectively. The proposed method has great potential in clinical spinal disease diagnoses and treatments.

A deep learning framework for prostate localization in cone beam CT-guided radiotherapy

PURPOSE

To develop a deep learning-based model for prostate planning target volume (PTV) localization on cone beam computed tomography (CBCT) to improve the workflow of CBCT-guided patient setup.

METHODS

A two-step task-based residual network (T2 RN) is proposed to automatically identify inherent landmarks in prostate PTV. The input to the T2 RN is the pretreatment CBCT images of the patient, and the output is the deep learning-identified landmarks in the PTV. To ensure robust PTV localization, the T2 RN model is trained by using over thousand sets of CT images with labeled landmarks, each of the CTs corresponds to a different scenario of patient position and/or anatomy distribution generated by synthetically changing the planning CT (pCT) image. The changes, including translation, rotation, and deformation, represent vast possible clinical situations of anatomy variations during a course of radiation therapy (RT). The trained patient-specific T2 RN model is tested by using 240 CBCTs from six patients. The testing CBCTs consists of 120 original CBCTs and 120 synthetic CBCTs. The synthetic CBCTs are generated by applying rotation/translation transformations to each of the original CBCT.

RESULTS

The systematic/random setup errors between the model prediction and the reference are found to be <0.25/2.46 mm and 0.14/1.41° in translation and rotation dimensions, respectively. Pearson's correlation coefficient between model prediction and the reference is higher than 0.94 in translation and rotation dimensions. The Bland-Altman plots show good agreement between the two techniques.

CONCLUSIONS

A novel T2 RN deep learning technique is established to localize the prostate PTV for RT patient setup. Our results show that highly accurate marker-less prostate setup is achievable by leveraging the state-of-the-art deep learning strategy.

Superpixel Region Merging Based on Deep Network for Medical Image Segmentation

Automatic and accurate semantic segmentation of pathological structures in medical images is challenging because of noisy disturbance, deformable shapes of pathology, and low contrast between soft tissues. Classical superpixel-based classification algorithms suffer from edge leakage due to complexity and heterogeneity inherent in medical images. Therefore, we propose a deep U-Net with superpixel region merging processing incorporated for edge enhancement to facilitate and optimize segmentation. Our approach combines three innovations: (1) different from deep learning--based image segmentation, the segmentation evolved from superpixel region merging via U-Net training getting rich semantic information, in addition to gray similarity; (2) a bilateral filtering module was adopted at the beginning of the network to eliminate external noise and enhance soft tissue contrast at edges of pathogy; and (3) a normalization layer was inserted after the convolutional layer at each feature scale, to prevent overfitting and increase the sensitivity to model parameters. This model was validated on lung CT, brain MR, and coronary CT datasets, respectively. Different superpixel methods and cross validation show the effectiveness of this architecture. The hyperparameter settings were empirically explored to achieve a good trade-off between the performance and efficiency, where a four-layer network achieves the best result in precision, recall, F-measure, and running speed. It was demonstrated that our method outperformed state-of-the-art networks, including FCN-16s, SegNet, PSPNet, DeepLabv3, and traditional U-Net, both quantitatively and qualitatively. Source code for the complete method is available at https://github.com/Leahnawho/Superpixel-network.

PROSTATE CANCER DETECTION USING HISTOPATHOLOGY IMAGES AND CLASSIFICATION USING IMPROVED RideNN

Medical industry reports prostate cancer as common and high among men and alarms the necessity for detecting prostate cancer for which the required morphology is extracted from the histopathology images. Commonly, the Gleason grading system remains a perfect factor for grading prostate cancer in men, but pathologists suffer from minute inter- and intra-observer variations. Thus, an automatic method for segmenting and classifying prostate cancer is modeled in this paper. The significance of the developed method is that the segmentation and classification are gland-oriented using the Color Space (CS) transformation and Salp Swarm Optimization Algorithm-based Rider Neural Network (SSA-RideNN). The gland region is considered as the morphology for cancer detection from which the maximal significant regions are extracted as features using multiple-kernel scale-invariant feature transform (MK-SIFT). Here, the RideNN classifier is trained optimally using the proposed Salp–Rider Algorithm (SRA), which is the integration of Salp Swarm Optimization Algorithm (SSA) and Rider Optimization Algorithm (ROA). The experimentation is performed using the histopathology images and the analysis based on sensitivity, accuracy, and specificity reveals that the proposed prostate cancer detection method acquired the maximal accuracy, sensitivity, and specificity of 0.8966, 0.8919, and 0.8596, respectively.

Iterative fully convolutional neural networks for automatic vertebra segmentation and identification

Precise segmentation and anatomical identification of the vertebrae provides the basis for automatic analysis of the spine, such as detection of vertebral compression fractures or other abnormalities. Most dedicated spine CT and MR scans as well as scans of the chest, abdomen or neck cover only part of the spine. Segmentation and identification should therefore not rely on the visibility of certain vertebrae or a certain number of vertebrae. We propose an iterative instance segmentation approach that uses a fully convolutional neural network to segment and label vertebrae one after the other, independently of the number of visible vertebrae. This instance-by-instance segmentation is enabled by combining the network with a memory component that retains information about already segmented vertebrae. The network iteratively analyzes image patches, using information from both image and memory to search for the next vertebra. To efficiently traverse the image, we include the prior knowledge that the vertebrae are always located next to each other, which is used to follow the vertebral column. The network concurrently performs multiple tasks, which are segmentation of a vertebra, regression of its anatomical label and prediction whether the vertebra is completely visible in the image, which allows to exclude incompletely visible vertebrae from further analyses. The predicted anatomical labels of the individual vertebrae are additionally refined with a maximum likelihood approach, choosing the overall most likely labeling if all detected vertebrae are taken into account. This method was evaluated with five diverse datasets, including multiple modalities (CT and MR), various fields of view and coverages of different sections of the spine, and a particularly challenging set of low-dose chest CT scans. For vertebra segmentation, the average Dice score was 94.9 ± 2.1% with an average absolute symmetric surface distance of 0.2 ± 10.1mm. The anatomical identification had an accuracy of 93%, corresponding to a single case with mislabeled vertebrae. Vertebrae were classified as completely or incompletely visible with an accuracy of 97%. The proposed iterative segmentation method compares favorably with state-of-the-art methods and is fast, flexible and generalizable.

Automatic detection System for Degenerative Disk and simulation for artificial disc replacement surgery in the spine

This paper presents an automatic detection system for the degenerative disc and simulates the artificial disc replacement surgery in the spine. It starts by visualizing the Digital Imaging and Communications in Medicine (DICOM) in three views and then identifies regions for the lumber or sacral, followed by applying an adaptive threshold and modified region growing techniques to separate the spine from the surrounding tissues, organs, and bones. Then, segment each vertebral in the spine using K-means clustering algorithm. Then, the distance of intervertebral space was computed to detect the degenerative disk automatically in the lumbar region. Finally, the replacement surgery simulation occurred to design the disc with exact dimensions and identifies the appropriate place that will be implanted in the spine. The proposed system was tested on ten 3D computed tomography (CT) images downloaded from the Cancer Imaging Archive (TCIA). It was accessible to implant the artificial disk instead of the degenerative disk in the space that was specified automatically by the proposed system. The experimental outcomes positively demonstrate that the intended system is efficacious in the detection of degenerative disc and replacement of artificial disc simulation.

Machine Learning Applications in Head and Neck Radiation Oncology: Lessons From Open-Source Radiomics Challenges

Radiomics leverages existing image datasets to provide non-visible data extraction via image post-processing, with the aim of identifying prognostic, and predictive imaging features at a sub-region of interest level. However, the application of radiomics is hampered by several challenges such as lack of image acquisition/analysis method standardization, impeding generalizability. As of yet, radiomics remains intriguing, but not clinically validated. We aimed to test the feasibility of a non-custom-constructed platform for disseminating existing large, standardized databases across institutions for promoting radiomics studies. Hence, University of Texas MD Anderson Cancer Center organized two public radiomics challenges in head and neck radiation oncology domain. This was done in conjunction with MICCAI 2016 satellite symposium using Kaggle-in-Class, a machine-learning and predictive analytics platform. We drew on clinical data matched to radiomics data derived from diagnostic contrast-enhanced computed tomography (CECT) images in a dataset of 315 patients with oropharyngeal cancer. Contestants were tasked to develop models for (i) classifying patients according to their human papillomavirus status, or (ii) predicting local tumor recurrence, following radiotherapy. Data were split into training, and test sets. Seventeen teams from various professional domains participated in one or both of the challenges. This review paper was based on the contestants' feedback; provided by 8 contestants only (47%). Six contestants (75%) incorporated extracted radiomics features into their predictive model building, either alone (n = 5; 62.5%), as was the case with the winner of the "HPV" challenge, or in conjunction with matched clinical attributes (n = 2; 25%). Only 23% of contestants, notably, including the winner of the "local recurrence" challenge, built their model relying solely on clinical data. In addition to the value of the integration of machine learning into clinical decision-making, our experience sheds light on challenges in sharing and directing existing datasets toward clinical applications of radiomics, including hyper-dimensionality of the clinical/imaging data attributes. Our experience may help guide researchers to create a framework for sharing and reuse of already published data that we believe will ultimately accelerate the pace of clinical applications of radiomics; both in challenge or clinical settings.

Radiomics and radiogenomics for precision radiotherapy

Imaging plays an important role in the diagnosis and staging of cancer, as well as in radiation treatment planning and evaluation of therapeutic response. Recently, there has been significant interest in extracting quantitative information from clinical standard-of-care images, i.e. radiomics, in order to provide a more comprehensive characterization of image phenotypes of the tumor. A number of studies have demonstrated that a deeper radiomic analysis can reveal novel image features that could provide useful diagnostic, prognostic or predictive information, improving upon currently used imaging metrics such as tumor size and volume. Furthermore, these imaging-derived phenotypes can be linked with genomic data, i.e. radiogenomics, in order to understand their biological underpinnings or further improve the prediction accuracy of clinical outcomes. In this article, we will provide an overview of radiomics and radiogenomics, including their rationale, technical and clinical aspects. We will also present some examples of the current results and some emerging paradigms in radiomics and radiogenomics for clinical oncology, with a focus on potential applications in radiotherapy. Finally, we will highlight the challenges in the field and suggest possible future directions in radiomics to maximize its potential impact on precision radiotherapy.

Combining deep learning with anatomical analysis for segmentation of the portal vein for liver SBRT planning

Automated segmentation of the portal vein (PV) for liver radiotherapy planning is a challenging task due to potentially low vasculature contrast, complex PV anatomy and image artifacts originated from fiducial markers and vasculature stents. In this paper, we propose a novel framework for automated segmentation of the PV from computed tomography (CT) images. We apply convolutional neural networks (CNNs) to learn the consistent appearance patterns of the PV using a training set of CT images with reference annotations and then enhance the PV in previously unseen CT images. Markov random fields (MRFs) were further used to smooth the results of the enhancement of the CNN enhancement and remove isolated mis-segmented regions. Finally, CNN-MRF-based enhancement was augmented with PV centerline detection that relied on PV anatomical properties such as tubularity and branch composition. The framework was validated on a clinical database with 72 CT images of patients scheduled for liver stereotactic body radiation therapy. The obtained accuracy of the segmentation was [Formula: see text] 0.83 and [Formula: see text] 1.08 mm in terms of the median Dice coefficient and mean symmetric surface distance, respectively, when segmentation is encompassed into the PV region of interest. The obtained results indicate that CNNs and anatomical analysis can be used for the accurate segmentation of the PV and potentially integrated into liver radiation therapy planning.