Fully automated segmentation of a hip joint using the patient-specific optimal thresholding and watershed algorithm

Two-stage multi-task deep learning framework for simultaneous pelvic bone segmentation and landmark detection from CT images

PURPOSE

Pelvic bone segmentation and landmark definition from computed tomography (CT) images are prerequisite steps for the preoperative planning of total hip arthroplasty. In clinical applications, the diseased pelvic anatomy usually degrades the accuracies of bone segmentation and landmark detection, leading to improper surgery planning and potential operative complications.

METHODS

This work proposes a two-stage multi-task algorithm to improve the accuracy of pelvic bone segmentation and landmark detection, especially for the diseased cases. The two-stage framework uses a coarse-to-fine strategy which first conducts global-scale bone segmentation and landmark detection and then focuses on the important local region to further refine the accuracy. For the global stage, a dual-task network is designed to share the common features between the segmentation and detection tasks, so that the two tasks mutually reinforce each other's performance. For the local-scale segmentation, an edge-enhanced dual-task network is designed for simultaneous bone segmentation and edge detection, leading to the more accurate delineation of the acetabulum boundary.

RESULTS

This method was evaluated via threefold cross-validation based on 81 CT images (including 31 diseased and 50 healthy cases). The first stage achieved DSC scores of 0.94, 0.97, and 0.97 for the sacrum, left and right hips, respectively, and an average distance error of 3.24 mm for the bone landmarks. The second stage further improved the DSC of the acetabulum by 5.42%, and this accuracy outperforms the state-of-the-arts (SOTA) methods by 0.63%. Our method also accurately segmented the diseased acetabulum boundaries. The entire workflow took ~ 10 s, which was only half of the U-Net run time.

CONCLUSION

Using the multi-task networks and the coarse-to-fine strategy, this method achieved more accurate bone segmentation and landmark detection than the SOTA method, especially for diseased hip images. Our work contributes to accurate and rapid design of acetabular cup prostheses.

MsgeCNN: Multiscale geometric embedded convolutional neural network for ONFH segmentation and grading

BACKGROUND

The incidence of osteonecrosis of the femoral head (ONFH) is increasing gradually, rapid and accurate grading of ONFH is critical. The existing Steinberg staging criteria grades ONFH according to the proportion of necrosis area to femoral head area.

PURPOSE

In the clinical practice, the necrosis region and femoral head region are mainly estimated by the observation and experience of doctor. This paper proposes a two-stage segmentation and grading framework, which can be used to segment the femoral head and necrosis, as well as to diagnosis.

METHODS

The core of the proposed two-stage framework is the multiscale geometric embedded convolutional neural network (MsgeCNN), which integrates geometric information into the training process and accurately segments the femoral head region. Then, the necrosis regions are segmented by the adaptive threshold method taking femoral head as the background. The area and proportion of the two are calculated to determine the grade.

RESULTS

The accuracy of the proposed MsgeCNN for femoral head segmentation is 97.73%, sensitivity is 91.17%, specificity is 99.40%, dice score is 93.34%. And the segmentation performance is better than the existing five segmentation algorithms. The diagnostic accuracy of the overall framework is 90.80%.

CONCLUSIONS

The proposed framework can accurately segment the femoral head region and the necrosis region. The area, proportion, and other pathological information of the framework output provide auxiliary strategies for subsequent clinical treatment.

Efficient cascaded V-net optimization for lower extremity CT segmentation validated using bone morphology assessment

Semantic segmentation of bone from lower extremity computerized tomography (CT) scans can improve and accelerate the visualization, diagnosis, and surgical planning in orthopaedics. However, the large field of view of these scans makes automatic segmentation using deep learning based methods challenging, slow and graphical processing unit (GPU) memory intensive. We investigated methods to more efficiently represent anatomical context for accurate and fast segmentation and compared these with state-of-the-art methodology. Six lower extremity bones from patients of two different datasets were manually segmented from CT scans, and used to train and optimize a cascaded deep learning approach. We varied the number of resolution levels, receptive fields, patch sizes, and number of V-net blocks. The best performing network used a multi-stage, cascaded V-net approach with 128³ -64³ -32³ voxel patches as input. The average Dice coefficient over all bones was 0.98 ± 0.01, the mean surface distance was 0.26 ± 0.12 mm and the 95th percentile Hausdorff distance 0.65 ± 0.28 mm. This was a significant improvement over the results of the state-of-the-art nnU-net, with only approximately 1/12th of training time, 1/3th of inference time and 1/4th of GPU memory required. Comparison of the morphometric measurements performed on automatic and manual segmentations showed good correlation (Intraclass Correlation Coefficient [ICC] >0.8) for the alpha angle and excellent correlation (ICC >0.95) for the hip-knee-ankle angle, femoral inclination, femoral version, acetabular version, Lateral Centre-Edge angle, acetabular coverage. The segmentations were generally of sufficient quality for the tested clinical applications and were performed accurately and quickly compared to state-of-the-art methodology from the literature.

A Review of CT-Based Fracture Risk Assessment with Finite Element Modeling and Machine Learning

PURPOSE OF REVIEW

We reviewed advances over the past 3 years in assessment of fracture risk based on CT scans, considering methods that use finite element models, machine learning, or a combination of both.

RECENT FINDINGS

Several studies have demonstrated that CT-based assessment of fracture risk, using finite element modeling or biomarkers derived from machine learning, is equivalent to currently used clinical tools. Phantomless calibration of CT scans for bone mineral density enables accurate measurements from routinely taken scans. This opportunistic use of CT scans for fracture risk assessment is facilitated by high-quality automated segmentation with deep learning, enabling workflows that do not require user intervention. Modeling of more realistic and diverse loading conditions, as well as improved modeling of fracture mechanisms, has shown promise to enhance our understanding of fracture processes and improve the assessment of fracture risk beyond the performance of current clinical tools. CT-based screening for fracture risk is effective and, by analyzing scans that were taken for other indications, could be used to expand the pool of people screened, therefore improving fracture prevention. Finite element modeling and machine learning both provide valuable tools for fracture risk assessment. Future approaches should focus on including more loading-related aspects of fracture risk.

Digital chain for pelvic tumor resection with 3D-printed surgical cutting guides

Surgical cutting guides are 3D-printed customized tools that help surgeons during complex surgeries. However, there does not seem to be any set methodology for designing these patient-specific instruments. Recent publications using pelvic surgical guides showed various designs with no clearly classified or standardized features. We, thus, developed a systematic digital chain for processing multimodal medical images (CT and MRI), designing customized surgical cutting guides, and manufacturing them using additive manufacturing. The aim of this study is to describe the steps in the conception of surgical cutting guides used in complex oncological bone tumor pelvic resection. We also analyzed the duration of the surgical cutting guide process and tested its ergonomics and usability with orthopedic surgeons using Sawbones models on simulated tumors. The original digital chain made possible a repeatable design of customized tools in short times. Preliminary testing on synthetic bones showed satisfactory results in terms of design usability. The four artificial tumors (Enneking I, Enneking II, Enneking III, and Enneking I+IV) were successfully resected from the Sawbones model using this digital chain with satisfactory ergonomic outcomes. This work validates a new digital chain conception and production of surgical cutting guides. Further works with quantitative margin assessments on anatomical subjects are needed to better assess the design implications of patient-specific surgical cutting guide instruments in pelvic tumor resections.

Accuracy and reliability analysis of a machine learning based segmentation tool for intertrochanteric femoral fracture CT

Introduction

Three-dimensional (3D) reconstruction of fracture fragments on hip Computed tomography (CT) may benefit the injury detail evaluation and preoperative planning of the intertrochanteric femoral fracture (IFF). Manually segmentation of bony structures was tedious and time-consuming. The purpose of this study was to propose an artificial intelligence (AI) segmentation tool to achieve semantic segmentation and precise reconstruction of fracture fragments of IFF on hip CTs.

Materials and Methods

A total of 50 labeled CT cases were manually segmented with Slicer 4.11.0. The ratio of training, validation and testing of the 50 labeled dataset was 33:10:7. A simplified V-Net architecture was adopted to build the AI tool named as IFFCT for automatic segmentation of fracture fragments. The Dice score, precision and sensitivity were computed to assess the segmentation performance of IFFCT. The 2D masks of 80 unlabeled CTs segmented by AI tool and human was further assessed to validate the segmentation accuracy. The femoral head diameter (FHD) was measured on 3D models to validate the reliability of 3D reconstruction.

Results

The average Dice score of IFFCT in the local test dataset for “proximal femur”, “fragment” and “distal femur” were 91.62%, 80.42% and 87.05%, respectively. IFFCT showed similar segmentation performance in cross-dataset, and was comparable to that of human expert in human-computer competition with significantly reduced segmentation time (p < 0.01). Significant differences were observed between 2D masks generated from semantic segmentation and conventional threshold-based segmentation (p < 0.01). The average FHD in the automatic segmentation group was 47.5 ± 4.1 mm (41.29∼56.59 mm), and the average FHD in the manual segmentation group was 45.9 ± 6.1 mm (40.34∼64.93 mm). The mean absolute error of FHDs in the two groups were 3.38 mm and 3.52 mm, respectively. No significant differences of FHD measurements were observed between the two groups (p > 0.05). All ICCs were greater than 0.8.

Conclusion

The proposed AI segmentation tool could effectively segment the bony structures from IFF CTs with comparable performance of human experts. The 2D masks and 3D models generated from automatic segmentation were effective and reliable, which could benefit the injury detail evaluation and preoperative planning of IFFs.

Automated bone healing evaluation: New approach to histomorphometric analysis

This study aimed to assess different approaches for bone healing evaluation on histological images and to introduce a new automatic evaluation method based on segmentation with distinct thresholds. We evaluated the hyperbaric oxygen therapy (HBO) effects on bone repair in type 1 diabetes mellitus rats. Twelve animals were divided into four groups (n = 3): non-diabetic, non-diabetic + HBO, diabetic, and diabetic + HBO. Diabetes was induced by intravenous administration of streptozotocin (50 mg/kg). Bone defects were created in femurs and HBO was immediately started at one session/day. After 7 days, the animals were euthanized, femurs were removed, demineralized, and embedded in paraffin. Histological sections were stained with hematoxylin and eosin (HE) and Mallory's trichrome (MT), and evaluated using three approaches: (1) conventional histomorphometric analysis (HE images) using a 144-point grid to quantify the bone matrix; (2) a semi-automatic method based on bone matrix segmentation to assess the bone matrix percentage (MT images); and (3) automatic approach, with the creation of a plug-in for ImageJ software. The time required to perform the analysis in each method was measured and subjected to Bland-Altman statistical analysis. All three methods were satisfactory for measuring bone formation and were not statistically different. The automatic approach reduced the working time compared to visual grid and semi-automated method (p < .01). Although histological evaluation of bone healing was performed successfully using all three methods, the novel automatic approach significantly shortened the time required for analysis and had high accuracy.

A deep learning-based approach to automatic proximal femur segmentation in quantitative CT images

Automatic CT segmentation of proximal femur has a great potential for use in orthopedic diseases, especially in the imaging-based assessments of hip fracture risk. In this study, we proposed an approach based on deep learning for the fast and automatic extraction of the periosteal and endosteal contours of proximal femur in order to differentiate cortical and trabecular bone compartments. A three-dimensional (3D) end-to-end fully convolutional neural network (CNN), which can better combine the information among neighbor slices and get more accurate segmentation results by 3D CNN, was developed for our segmentation task. The separation of cortical and trabecular bones derived from the QCT software MIAF-Femur was used as the segmentation reference. Two models with the same network structures were trained, and they achieved a dice similarity coefficient (DSC) of 97.82% and 96.53% for the periosteal and endosteal contours, respectively. Compared with MIAF-Femur, it takes half an hour to segment a case, and our CNN model takes a few minutes. To verify the excellent performance of our model for proximal femoral segmentation, we measured the volumes of different parts of the proximal femur and compared it with the ground truth, and the relative errors of femur volume between predicted result and ground truth are all less than 5%. This approach will be expected helpful to measure the bone mineral densities of cortical and trabecular bones, and to evaluate the bone strength based on FEA.

Utility of a novel integrated deep convolutional neural network for the segmentation of hip joint from computed tomography images in the preoperative planning of total hip arthroplasty

Purpose

Preoperative three-dimensional planning is important for total hip arthroplasty. To simulate the placement of joint implants on computed tomography (CT), pelvis and femur must be segmented. Accurate and rapid segmentation of the hip joint is challenging. This study aimed to develop a novel deep learning network, named Changmugu Net (CMG Net), which could achieve accurate segmentation of the femur and pelvis.

Methods

The overall deep neural network architecture of CMG Net employed three interrelated modules. CMG Net included the 2D U-net to separate the bony and soft tissues. The modular hierarchy method was used for the main femur segmentation to achieve better performance. A layer classifier was adopted to localise femur layers among a series of CT scan images. The first module was a modified 2D U-net, which separated bony and soft tissues; it provided intermediate supervision for the main femur segmentation. The second module was the main femur segmentation, which was used to distinguish the femur from the acetabulum. The third module was the layer classifier, which served as a post-processor for the second module.

Results

There was a much greater overlap in accuracy results with the “gold standard” segmentation than with competing networks. The dice overlap coefficient was 93.55% ± 5.57%; the mean surface distance was 1.34 ± 0.24 mm, and the Hausdorff distance was 4.19 ± 1.04 mm in the normal and diseased hips, which indicated greater accuracy than the other four competing networks. Moreover, the mean segmentation time of CMG Net was 25.87 ± 2.73 s, which was shorter than the times of the other four networks.

Conclusions

The prominent segmentation accuracy and run-time of CMG Net suggest that it is a reliable method for clinicians to observe anatomical structures of the hip joints, even in severely diseased cases.

Multi-landmark environment analysis with reinforcement learning for pelvic abnormality detection and quantification

Morphological abnormalities of the femoroacetabular (hip) joint are among the most common human musculoskeletal disorders and often develop asymptomatically at early easily treatable stages. In this paper, we propose an automated framework for landmark-based detection and quantification of hip abnormalities from magnetic resonance (MR) images. The framework relies on a novel idea of multi-landmark environment analysis with reinforcement learning. In particular, we merge the concepts of the graphical lasso and Morris sensitivity analysis with deep neural networks to quantitatively estimate the contribution of individual landmark and landmark subgroup locations to the other landmark locations. Convolutional neural networks for image segmentation are utilized to propose the initial landmark locations, and landmark detection is then formulated as a reinforcement learning (RL) problem, where each landmark-agent can adjust its position by observing the local MR image neighborhood and the locations of the most-contributive landmarks. The framework was validated on T1-, T2- and proton density-weighted MR images of 260 patients with the aim to measure the lateral center-edge angle (LCEA), femoral neck-shaft angle (NSA), and the anterior and posterior acetabular sector angles (AASA and PASA) of the hip, and derive the quantitative abnormality metrics from these angles. The framework was successfully tested using the UNet and feature pyramid network (FPN) segmentation architectures for landmark proposal generation, and the deep Q-network (DeepQN), deep deterministic policy gradient (DDPG), twin delayed deep deterministic policy gradient (TD3), and actor-critic policy gradient (A2C) RL networks for landmark position optimization. The resulting overall landmark detection error of 1.5 mm and angle measurement error of 1.4° indicates a superior performance in comparison to existing methods. Moreover, the automatically estimated abnormality labels were in 95% agreement with those generated by an expert radiologist.

Image-based finite-element modeling of the human femur

Fracture is considered a critical clinical endpoint in skeletal pathologies including osteoporosis and bone metastases. However, current clinical guidelines are limited with respect to identifying cases at high risk of fracture, as they do not account for many mechanical determinants that contribute to bone fracture. Improving fracture risk assessment is an important area of research with clear clinical relevance. Patient-specific numerical musculoskeletal models generated from diagnostic images are widely used in biomechanics research and may provide the foundation for clinical tools used to quantify fracture risk. However, prior to clinical translation, in vitro validation of predictions generated from such numerical models is necessary. Despite adopting radically different models, in vitro validation of image-based finite element (FE) models of the proximal femur (predicting strains and failure loads) have shown very similar, encouraging levels of accuracy. The accuracy of such in vitro models has motivated their application to clinical studies of osteoporotic and metastatic fractures. Such models have demonstrated promising but heterogeneous results, which may be explained by the lack of a uniform strategy with respect to FE modeling of the human femur. This review aims to critically discuss the state of the art of image-based femoral FE modeling strategies, highlighting principal features and differences among current approaches. Quantitative results are also reported with respect to the level of accuracy achieved from in vitro evaluations and clinical applications and are used to motivate the adoption of a standardized approach/workflow for image-based FE modeling of the femur.

Hip-Joint CT Image Segmentation Based on Hidden Markov Model with Gauss Regression Constraints

Hip-joint CT images have low organizational contrast, irregular shape of boundaries and image noises. Traditional segmentation algorithms often require manual intervention or introduction of some prior information, which results in low efficiency and is unable to meet clinical needs. In order to overcome the sensitivity of classical fuzzy clustering image segmentation algorithm to image noise, this paper proposes a fuzzy clustering image segmentation algorithm combining Gaussian regression model (GRM) and hidden Markov random field (HMRF). The algorithm uses the prior information to regularize the objective function of the fuzzy C-means, and then improves it with KL information. The HMRF model establishes the neighborhood relationship of the label field by prior probability, while CRM model establishes the neighborhood relationship of feature field on the basis of the consistency between the central pixel label and its neighborhood pixel label. The experimental results show that the proposed algorithm has high segmentation accuracy.

Automated segmentation of cortical and trabecular bone to generate finite element models for femoral bone mechanics

Finite element (FE) models based on quantitative computed tomography (CT) images are better predictors of bone strength than conventional areal bone mineral density measurements. However, FE models require manual segmentation of the femur, which is not clinically applicable. This study developed a method for automated FE analyses from clinical CT images. Clinical in-vivo CT images of 13 elderly female subjects were collected to evaluate the method. Secondly, proximal cadaver femurs were harvested and imaged with clinical CT (N = 17). Of these femurs, 14 were imaged with µCT and three had earlier been tested experimentally in stance-loading, while collecting surface deformations with digital image correlation. Femurs were segmented from clinical CT images using an automated method, based on the segmentation tool Stradwin. The method automatically distinguishes trabecular and cortical bone, corrects partial volume effect and generates input for FE analysis. The manual and automatic segmentations agreed within about one voxel for in-vivo subjects (0.99 ± 0.23 mm) and cadaver femurs (0.21 ± 0.07 mm). The strains from the FE predictions closely matched with the experimentally measured strains (R² = 0.89). The method can automatically generate meshes suitable for FE analysis. The method may bring us one step closer to enable clinical usage of patient-specific FE analyses.

A new scheme for automatic 2D detection of spheric and aspheric femoral heads: A case study on coronal MR images of bilateral hip joints of patients with Legg-Calve-Perthes disease

BACKGROUND AND OBJECTIVE

In orthopaedics, hip joint and circular structure of the femoral head can be distorted by a wide variety of hip joint disorders. Hence, automatic detection and segmentation of the femoral head is an important issue in the studies of computerized hip joint segmentation, quantification and assessment. In the proposed study, we need to detect the center coordinates and radius of spheric and aspheric femoral heads automatically in bilateral hip magnetic resonance (MR) images using a new scheme.

METHODS

This paper presents a new two-level scheme based on the Circular Hough Transform (CHT) and Integro-differential Operator (IDO) to detect the spheric and aspheric femoral heads in MR images in 2D. Initially, MR slices are divided vertically into two equal halves to automatically separate the hip joints and Canny's edge detection method is performed on each of the halves to obtain edge images. Then, healthy and pathological femoral heads are detected by performing the CHT over edge images in the first stage of proposed scheme. In the second stage, femoral head circle detected with CHT is fine tuned by performing Daugman's IDO.

RESULTS

Performance evaluations of the proposed femoral head detection scheme were carried out on healthy and pathological hip joints in 24 coronal MR image sections belong to 13 patients with Legg-Calve-Perthes Disease (LCPD). In performance evaluations on 24 healthy and 24 pathological hip joints, Root Mean Square Error (RMSE) and Dice Similarity Coefficient (DSC) values were measured for automatically detected femoral heads. We observed 1.96 mm. (std. 1.21 mm.) mean RMSE for center coordinates, 1.45 mm. (std. 1.39 mm) mean RMSE for radii, 0.8978 (std. 0.0733) mean DSC on healthy femoral heads and 3.56 mm. (std. 3.19 mm.) mean RMSE for center coordinates, 1.56 mm. (std. 1.33 mm.) mean RMSE for radii, 0.8529 (std. 0.0927) mean DSC on pathological femoral heads.

CONCLUSIONS

Proposed femoral head detection scheme has promising results for the detection and the segmentation of the spheric and aspheric femoral heads and also has a potential to be used in detection of the other anatomical structures having a circular shape.

Patient-Specific Phantomless Estimation of Bone Mineral Density and Its Effects on Finite Element Analysis Results: A Feasibility Study

Objectives

This study proposes a regression model for the phantomless Hounsfield units (HU) to bone mineral density (BMD) conversion including patient physical factors and analyzes the accuracy of the estimated BMD values.

Methods

The HU values, BMDs, circumferences of the body, and cross-sectional areas of bone were measured from 39 quantitative computed tomography images of L2 vertebrae and hips. Then, the phantomless HU-to-BMD conversion was derived using a multiple linear regression model. For the statistical analysis, the correlation between the estimated BMD values and the reference BMD values was evaluated using Pearson's correlation test. Voxelwise BMD and finite element analysis (FEA) results were analyzed in terms of root-mean-square error (RMSE) and strain energy density, respectively.

Results

The HU values and circumferences were statistically significant (p < 0.05) for the lumbar spine, whereas only the HU values were statistically significant (p < 0.05) for the proximal femur. The BMD values estimated using the proposed HU-to-BMD conversion were significantly correlated with those measured using the reference phantom: Pearson's correlation coefficients of 0.998 and 0.984 for the lumbar spine and proximal femur, respectively. The RMSEs of the estimated BMD values for the lumbar spine and hip were 4.26 ± 0.60 (mg/cc) and 8.35 ± 0.57 (mg/cc), respectively. The errors of total strain energy were 1.06% and 0.91%, respectively.

Conclusions

The proposed phantomless HU-to-BMD conversion demonstrates the potential of precisely estimating BMD values from CT images without the reference phantom and being utilized as a viable tool for FEA-based quantitative assessment using routine CT images.

Computational study of estimating 3D trabecular bone microstructure for the volume of interest from CT scan data

Inspired by the self-optimizing capabilities of bone, a new concept of bone microstructure reconstruction has been recently introduced by using 2D synthetic skeletal images. As a preliminary clinical study, this paper proposes a topology optimization-based method that can estimate 3D trabecular bone microstructure for the volume of interest (VOI) from 3D computed tomography (CT) scan data with enhanced computational efficiency and phenomenological accuracy. For this purpose, a localized finite element (FE) model is constructed by segmenting a target bone from CT scan data and determining the physiological local loads for the VOI. Then, topology optimization is conducted with multiresolution bone mineral density (BMD) deviation constraints to preserve the patient-specific spatial bone distribution obtained from the CT scan data. For the first time, to our knowledge, this study has demonstrated that 60-μm resolution trabecular bone images can be reconstructed from 600-μm resolution CT scan data (a 62-year-old woman with no metabolic bone disorder) for the 4 VOIs in the proximal femur. The reconstructed trabecular bone includes the characteristic trabecular patterns and has morphometric indices that are in good agreement with the anatomical data in the literature. As for computational efficiency, the localization for the VOI reduces the number of FEs by 99%, compared with that of the full FE model. Compared with the previous single-resolution BMD deviation constraint, the proposed multiresolution BMD deviation constraints enable at least 65% and 47% reductions in the number of iterations and computing time, respectively. These results demonstrate the clinical feasibility and potential of the proposed method.