Integrating spatial configuration into heatmap regression based CNNs for landmark localization

Central loss guides coordinated Transformer for reliable anatomical landmark detection

Heatmap-based anatomical landmark detection is still facing two unresolved challenges: (1) inability to accurately evaluate the distribution of heatmap; (2) inability to effectively exploit global spatial structure information. To address the computational inability challenge, we propose a novel position-aware and sample-aware central loss. Specifically, our central loss can absorb position information, enabling accurate evaluation of the heatmap distribution. More advanced is that our central loss is sample-aware, which can adaptively distinguish easy and hard samples and make the model more focused on hard samples while solving the challenge of extreme imbalance between landmarks and non-landmarks. To address the challenge of ignoring structure information, a Coordinated Transformer, called CoorTransformer, is proposed, which establishes long-range dependencies under the guidance of landmark coordinate information, making the attention more focused on the sparse landmarks while taking advantage of global spatial structure. Furthermore, CoorTransformer can speed up convergence, effectively avoiding the defect that Transformers have difficulty converging in sparse representation learning. Using the advanced CoorTransformer and central loss, we propose a generalized detection model that can handle various scenarios, inherently exploiting the underlying relationship between landmarks and incorporating rich structural knowledge around the target landmarks. We analyzed and evaluated CoorTransformer and central loss on three challenging landmark detection tasks. The experimental results show that our CoorTransformer outperforms state-of-the-art methods, and the central loss significantly improves the model's performance with p-values <0.05. The source code of this work is available at the GitHub repository.

The biomedical applications of artificial intelligence: an overview of decades of research

A significant area of computer science called artificial intelligence (AI) is successfully applied to the analysis of intricate biological data and the extraction of substantial associations from datasets for a variety of biomedical uses. AI has attracted significant interest in biomedical research due to its features: (i) better patient care through early diagnosis and detection; (ii) enhanced workflow; (iii) lowering medical errors; (v) lowering medical costs; (vi) reducing morbidity and mortality; (vii) enhancing performance; (viii) enhancing precision; and (ix) time efficiency. Quantitative metrics are crucial for evaluating AI implementations, providing insights, enabling informed decisions, and measuring the impact of AI-driven initiatives, thereby enhancing transparency, accountability, and overall impact. The implementation of AI in biomedical fields faces challenges such as ethical and privacy concerns, lack of awareness, technology unreliability, and professional liability. A brief discussion is given of the AI techniques, which include Virtual screening (VS), DL, ML, Hidden Markov models (HMMs), Neural networks (NNs), Generative models (GMs), Molecular dynamics (MD), and Structure-activity relationship (SAR) models. The study explores the application of AI in biomedical fields, highlighting its enhanced predictive accuracy, treatment efficacy, diagnostic efficiency, faster decision-making, personalised treatment strategies, and precise medical interventions.

Integrating anatomy and electrophysiology in the healthy human heart: Insights from biventricular statistical shape analysis using universal coordinates

A cardiac digital twin is a virtual replica of a patient-specific heart, mimicking its anatomy and physiology. A crucial step of building a cardiac digital twin is anatomical twinning, where the computational mesh of the digital twin is tailored to the patient-specific cardiac anatomy. In a number of studies, the effect of anatomical variation on clinically relevant functional measurements like electrocardiograms (ECGs) is investigated, using computational simulations. While such a simulation environment provides researchers with a carefully controlled ground truth, the impact of anatomical differences on functional measurements in real-world patients remains understudied. In this study, we develop a biventricular statistical shape model and use it to quantify the effect of biventricular anatomy on ECG-derived and demographic features, providing novel insights for the development of digital twins of cardiac electrophysiology. To this end, a dataset comprising high-resolution cardiac CT scans from 271 healthy individuals, including athletes, is utilized. Furthermore, a novel, universal, ventricular coordinate-based method is developed to establish lightweight shape correspondence. The performance of the shape model is rigorously established, focusing on its dimensionality reduction capabilities and the training data requirements. The most important variability in healthy ventricles captured by the model is their size, followed by their elongation. These anatomical factors are found to significantly correlate with ECG-derived and demographic features. Additionally, a comprehensive synthetic cohort is made available, featuring ready-to-use biventricular meshes with fiber structures and anatomical region annotations. These meshes are well-suited for electrophysiological simulations.

Deep Learning-enhanced Opportunistic Osteoporosis Screening in Ultralow-Voltage (80 kV) Chest CT: A Preliminary Study

RATIONALE AND OBJECTIVES

To explore the feasibility of deep learning (DL)-enhanced, fully automated bone mineral density (BMD) measurement using the ultralow-voltage 80 kV chest CT scans performed for lung cancer screening.

MATERIALS AND METHODS

This study involved 987 patients who underwent 80 kV chest and 120 kV lumbar CT from January to July 2024. Patients were collected from six CT scanners and divided into the training, validation, and test sets 1 and 2 (561: 177: 112: 137). Four convolutional neural networks (CNNs) were employed for automated segmentation (3D VB-Net and SCN), region of interest extraction (3D VB-Net), and BMD calculation (DenseNet and ResNet) of the target vertebrae (T12-L2). The BMD values of T12-L2 were obtained using 80 and 120 kV quantitative CT (QCT), the latter serving as the standard reference. Linear regression and Bland-Altman analyses were used to compare BMD values between 120 kV QCT and 80 kV CNNs, and between 120 kV QCT and 80 kV QCT. Receiver operating characteristic curve analysis was used to assess the diagnostic performance of the 80 kV CNNs and 80 kV QCT for osteoporosis and low BMD from normal BMD.

RESULTS

Linear regression and Bland-ltman analyses revealed a stronger correlation (R2=0.991-0.998 and 0.990-0.991, P<0.001) and better agreement (mean error, -1.36 to 1.62 and 1.72 to 2.27 mg/cm3; 95% limits of agreement, -9.73 to 7.01 and -5.71 to 10.19mg/cm3) for BMD between 120 kV QCT and 80 kV CNNs than between 120 kV QCT and 80 kV QCT. The areas under the curve of the 80 kV CNNs and 80 kV QCT in detecting osteoporosis and low BMD were 0.997-1.000 and 0.997-0.998, and 0.998-1.000 and 0.997, respectively.

CONCLUSION

The DL method could achieve fully automated BMD calculation for opportunistic osteoporosis screening with high accuracy using ultralow-voltage 80 kV chest CT performed for lung cancer screening.

Eigenhearts: Cardiac diseases classification using eigenfaces approach

In the realm of cardiovascular medicine, medical imaging plays a crucial role in accurately classifying cardiac diseases and making precise diagnoses. However, the integration of data science techniques in this field presents significant challenges, as it requires a large volume of images, while ethical constraints, high costs, and variability in imaging protocols limit data acquisition. As a consequence, it is necessary to investigate different avenues to overcome this challenge. In this contribution, we offer an innovative tool to conquer this limitation. In particular, we delve into the application of a well recognized method known as the eigenfaces approach to classify cardiac diseases. This approach was originally motivated for efficiently representing pictures of faces using principal component analysis, which provides a set of eigenvectors (aka eigenfaces), explaining the variation between face images. Given its effectiveness in face recognition, we sought to evaluate its applicability to more complex medical imaging datasets. In particular, we integrate this approach with convolutional neural networks to classify echocardiography images taken from mice in five distinct cardiac conditions (healthy, diabetic cardiomyopathy, myocardial infarction, obesity and TAC hypertension). The results show a substantial and noteworthy enhancement when employing the singular value decomposition for pre-processing, with classification accuracy increasing by approximately 50%.

Torg-Pavlov ratio qualification to diagnose developmental cervical spinal stenosis based on HRViT neural network

BACKGROUND

Developing computer-assisted methods to measure the Torg-Pavlov ratio (TPR), defined as the ratio of the sagittal diameter of the cervical spinal canal to the sagittal diameter of the corresponding vertebral body on lateral radiographs, can reduce subjective influence and speed up processing. The TPR is a critical diagnostic parameter for developmental cervical spinal stenosis (DCSS), as it normalizes variations in radiographic magnification and provides a cost-effective alternative to CT/MRI in resource-limited settings. No study focusing on automatic measurement was reported. The aim was to develop a deep learning-based model for automatically measuring the TPR, and then to establish the distribution of asymptomatic Chinese TPR.

METHODS

A total of 1623 lateral cervical X-ray images from normal individuals were collected. 1466 and 157 images were used as the training dataset and testing dataset, respectively. We adopted a neural network called High-Resolution Vision Transformer (HRViT), which was trained on the annotated X-ray image dataset to automatically locate the landmarks and calculate the TPR. The accuracy of the TPR measurement was evaluated using mean absolute error (MAE), intra-class correlation coefficient (ICC), r value and Bland-Altman plot.

RESULTS

The TPR at C2-C7 was 1.26, 0.92, 0.90, 0.93, 0.92, and 0.89, respectively. The MAE between HRViT and surgeon R1 was 0.01, between surgeon R1 and surgeon R2 was 0.17, between surgeon R1 and surgeon R3 was 0.17. The accuracy of HRViT for DCSS diagnosis was 84.1%, which was greatly higher than those of both surgeon R2 (57.3%) and surgeon R3 (56.7%). The consistency of TPR measurements was 0.77-0.9 (ICC) and 0.78-0.9 (r value) between HRViT and surgeon R1.

CONCLUSIONS

We have explored a deep-learning algorithm for automated measurement of the TPR on cervical lateral radiographs to diagnose DCSS, which had outstanding performance comparable to clinical senior doctors.

Automatic identification of hard and soft tissue landmarks in cone-beam computed tomography via deep learning with diversity datasets: a methodological study

BACKGROUND

Manual landmark detection in cone beam computed tomography (CBCT) for evaluating craniofacial structures relies on medical expertise and is time-consuming. This study aimed to apply a new deep learning method to predict and locate soft and hard tissue craniofacial landmarks on CBCT in patients with various types of malocclusion.

METHODS

A total of 498 CBCT images were collected. Following the calibration procedure, two experienced clinicians identified 43 landmarks in the x-, y-, and z-coordinate planes on the CBCT images using Checkpoint Software, creating the ground truth by averaging the landmark coordinates. To evaluate the accuracy of our algorithm, we determined the mean absolute error along the x-, y-, and z-axes and calculated the mean radial error (MRE) between the reference landmark and predicted landmark, as well as the successful detection rate (SDR).

RESULTS

Each landmark prediction took approximately 4.2 s on a conventional graphics processing unit. The mean absolute error across all coordinates was 0.74 mm. The overall MRE for the 43 landmarks was 1.76 ± 1.13 mm, and the SDR was 60.16%, 91.05%, and 97.58% within 2-, 3-, and 4-mm error ranges of manual marking, respectively. The average MRE of the hard tissue landmarks (32/43) was 1.73 mm, while that for soft tissue landmarks (11/43) was 1.84 mm.

CONCLUSIONS

Our proposed algorithm demonstrates a clinically acceptable level of accuracy and robustness for automatic detection of CBCT soft- and hard-tissue landmarks across all types of malformations. The potential for artificial intelligence to assist in identifying three dimensional-CT landmarks in routine clinical practice and analysing large datasets for future research is promising.

Automatic segmentation and landmark detection of 3D CBCT images using semi supervised learning for assisting orthognathic surgery planning

Patients with abnormal relative position of the upper and lower jaws (the main part of the facial bones) require orthognathic surgery to improve the occlusal relationship and facial appearance. However, in addition to the retraction and protrusion of the maxillomandibular advancement, these patients may also develop asymmetry. This study aims to use a semi-supervised learning method to demonstrate the maxillary and mandible retraction, protrudation and asymmetry of patients before orthognathic surgery through automatic segmentation of 3D cone beam computed tomography (CBCT) images and landmark detection, so as to provide help for the preoperative planning of orthognathic surgery. Among them, the dice of the semi-supervised algorithm adopted in this study reached 93.41 and 96.89% in maxillary and mandibular segmentation tasks, and the average error of landmark detection tasks reached 1.908 ± 1.166 mm, both of which were superior to the full-supervised algorithm with the same data volume annotation. Therefore, we propose that the method can be applied in a clinical setting to assist surgeons in preoperative planning for orthognathic surgery.

Deep learning-based pelvimetry in pelvic MRI volumes for pre-operative difficulty assessment of total mesorectal excision

BACKGROUND

Specific pelvic bone dimensions have been identified as predictors of total mesorectal excision (TME) difficulty and outcomes. However, manual measurement of these dimensions (pelvimetry) is labor intensive and thus, anatomic criteria are not included in the pre-operative difficulty assessment. In this work, we propose an automated workflow for pelvimetry based on pre-operative magnetic resonance imaging (MRI) volumes.

METHODS

We implement a deep learning-based framework to measure the predictive pelvic dimensions automatically. A 3D U-Net takes a sagittal T2-weighted MRI volume as input and determines five anatomic landmark locations: promontorium, S3-vertebrae, coccyx, dorsal, and cranial part of the os pubis. The landmarks are used to quantify the lengths of the pelvic inlet, outlet, depth, and the angle of the sacrum. For the development of the network, we used MRI volumes from 1707 patients acquired in eight TME centers. The automated landmark localization and pelvic dimensions measurements are assessed by comparison with manual annotation.

RESULTS

A center-stratified fivefold cross-validation showed a mean landmark localization error of 5.6 mm. The inter-observer variation for manual annotation was 3.7 ± 8.4 mm. The automated dimension measurements had a Spearman correlation coefficient ranging between 0.7 and 0.87.

CONCLUSION

To our knowledge, this is the first study to automate pelvimetry in MRI volumes using deep learning. Our framework can measure the pelvic dimensions with high accuracy, enabling the extraction of metrics that facilitate a pre-operative difficulty assessment of the TME.

A Projective-Geometry-Aware Network for 3D Vertebra Localization in Calibrated Biplanar X-Ray Images

Current Deep Learning (DL)-based methods for vertebra localization in biplanar X-ray images mainly focus on two-dimensional (2D) information and neglect the projective geometry, limiting the accuracy of 3D navigation in X-ray-guided spine surgery. A 3D vertebra localization method from calibrated biplanar X-ray images is highly desired to address the problem. In this study, a projective-geometry-aware network for localizing 3D vertebrae in calibrated biplanar X-ray images, referred to as ProVLNet, is proposed. The network design of ProVLNet features three components: a Siamese 2D feature extractor to extract local appearance features from the biplanar X-ray images, a spatial alignment fusion module to incorporate the projective geometry in fusing the extracted 2D features in 3D space, and a 3D landmark regression module to regress the 3D coordinates of the vertebrae from the 3D fused features. Evaluated on two typical and challenging datasets acquired from the lumbar and the thoracic spine, ProVLNet achieved an identification rate of 99.53% and 98.98% and a point-to-point error of 0.64 mm and 1.38 mm, demonstrating superior performance of our proposed approach over the state-of-the-art (SOTA) methods.

Semantics and instance interactive learning for labeling and segmentation of vertebrae in CT images

Automatically labeling and segmenting vertebrae in 3D CT images compose a complex multi-task problem. Current methods progressively conduct vertebra labeling and semantic segmentation, which typically include two separate models and may ignore feature interaction among different tasks. Although instance segmentation approaches with multi-channel prediction have been proposed to alleviate such issues, their utilization of semantic information remains insufficient. Additionally, another challenge for an accurate model is how to effectively distinguish similar adjacent vertebrae and model their sequential attribute. In this paper, we propose a Semantics and Instance Interactive Learning (SIIL) paradigm for synchronous labeling and segmentation of vertebrae in CT images. SIIL models semantic feature learning and instance feature learning, in which the former extracts spinal semantics and the latter distinguishes vertebral instances. Interactive learning involves semantic features to improve the separability of vertebral instances and instance features to help learn position and contour information, during which a Morphological Instance Localization Learning (MILL) module is introduced to align semantic and instance features and facilitate their interaction. Furthermore, an Ordinal Contrastive Prototype Learning (OCPL) module is devised to differentiate adjacent vertebrae with high similarity (via cross-image contrastive learning), and simultaneously model their sequential attribute (via a temporal unit). Extensive experiments on several datasets demonstrate that our method significantly outperforms other approaches in labeling and segmenting vertebrae. Our code is available at https://github.com/YuZhang-SMU/Vertebrae-Labeling-Segmentation.

A fully automatic fiducial detection and correspondence establishing method for online C-arm calibration

PURPOSE

Online C-arm calibration with a mobile fiducial cage plays an essential role in various image-guided interventions. However, it is challenging to develop a fully automatic approach, which requires not only an accurate detection of fiducial projections but also a robust 2D-3D correspondence establishment.

METHODS

We propose a novel approach for online C-arm calibration with a mobile fiducial cage. Specifically, a novel mobile calibration cage embedded with 16 fiducials is designed, where the fiducials are arranged to form 4 line patterns with different cross-ratios. Then, an auto-context-based detection network (ADNet) is proposed to perform an accurate and robust detection of 2D projections of those fiducials in acquired C-arm images. Subsequently, we present a cross-ratio consistency-based 2D-3D correspondence establishing method to automatically match the detected 2D fiducial projections with those 3D fiducials, allowing for an accurate online C-arm calibration.

RESULTS

We designed and conducted comprehensive experiments to evaluate the proposed approach. For automatic detection of 2D fiducial projections, the proposed ADNet achieved a mean point-to-point distance of 0.65 ± 1.33 pixels. Additionally, the proposed C-arm calibration approach achieved a mean re-projection error of 1.01 ± 0.63 pixels and a mean point-to-line distance of 0.22 ± 0.12  mm. When the proposed C-arm calibration approach was applied to downstream tasks involving landmark and surface model reconstruction, sub-millimeter accuracy was achieved.

CONCLUSION

In summary, we developed a novel approach for online C-arm calibration. Both qualitative and quantitative results of comprehensive experiments demonstrated the accuracy and robustness of the proposed approach. Our approach holds potentials for various image-guided interventions.

Anatomical prior-based vertebral landmark detection for spinal disorder diagnosis

As one of fundamental ways to interpret spine images, detection of vertebral landmarks is an informative prerequisite for further diagnosis and management of spine disorders such as scoliosis and fractures. Most existing machine learning-based methods for automatic vertebral landmark detection suffer from overlapping landmarks or abnormally long distances between nearby landmarks against anatomical priors, and thus lack sufficient reliability and interpretability. To tackle the problem, this paper systematically utilizes anatomical prior knowledge in vertebral landmark detection. We explicitly formulate anatomical priors of the spine, related to distances among vertebrae and spatial order within the spine, and integrate these geometrical constraints within training loss, inference procedure, and evaluation metrics. First, we introduce an anatomy-constraint loss to regularize the training process with the aforementioned contextual priors explicitly. Second, we propose a simple-yet-effective anatomy-aided inference procedure by employing sequential prediction rather than a parallel counterpart. Third, we provide novel anatomy-related metrics to quantitatively evaluate to which extent landmark predictions follow the anatomical priors, as is not reflected within the widely-used landmark localization error metric. We employ the localization framework on 1410 anterior-posterior radiographic images. Compared with competitive baseline models, we achieve superior landmark localization accuracy and comparable Cobb angle estimation for scoliosis assessment. Ablation studies demonstrate the effectiveness of designed components on the decrease of localization error and improvement of anatomical plausibility. Additionally, we exhibit effective generalization performance by transferring our detection method onto sagittal 2-D slices of CT scans and boost the performance of downstream compression fracture classification at vertebra-level.

IGU-Aug: Information-Guided Unsupervised Augmentation and Pixel-Wise Contrastive Learning for Medical Image Analysis

Contrastive learning (CL) is a form of self-supervised learning and has been widely used for various tasks. Different from widely studied instance-level contrastive learning, pixel-wise contrastive learning mainly helps with pixel-wise dense prediction tasks. The counterpart to an instance in instance-level CL is a pixel, along with its neighboring context, in pixel-wise CL. Aiming to build better feature representation, there is a vast literature about designing instance augmentation strategies for instance-level CL; but there is little similar work on pixel augmentation for pixel-wise CL with a pixel granularity. In this paper, we attempt to bridge this gap. We first classify a pixel into three categories, namely low-, medium-, and high-informative, based on the information quantity the pixel contains. We then adaptively design separate augmentation strategies for each category in terms of augmentation intensity and sampling ratio. Extensive experiments validate that our information-guided pixel augmentation strategy succeeds in encoding more discriminative representations and surpassing other competitive approaches in unsupervised local feature matching. Furthermore, our pretrained model improves the performance of both one-shot and fully supervised models. To the best of our knowledge, we are the first to propose a pixel augmentation method with a pixel granularity for enhancing unsupervised pixel-wise contrastive learning. Code is available at https://github.com/Curli-quan/IGU-Aug.

Cephalometric landmark annotation using transfer learning: Detectron2 and YOLOv8 baselines on a diverse cephalometric image dataset

BACKGROUND

Cephalometric landmark annotation is a key challenge in radiographic analysis, requiring automation due to its time-consuming process and inherent subjectivity. This study investigates the application of advanced transfer learning techniques to enhance the accuracy of anatomical landmarks in cephalometric images, which is a vital aspect of orthodontic diagnosis and treatment planning.

METHODS

We assess the suitability of transfer learning methods by employing state-of-the-art pose estimation models. The first framework is Detectron2, with two baselines featuring different ResNet backbone architectures: rcnn_R_50_FPN_3x and rcnn_R_101_FPN_3x. The second framework is YOLOv8, with three variants reflecting different network sizes: YOLOv8s-pose, YOLOv8m-pose, and YOLOv8l-pose. These pose estimation models are adopted for the landmark annotation task. The models are trained and evaluated on the DiverseCEPH19 dataset, comprising 1692 radiographic images with 19 landmarks, and their performance is analyzed across various images categories within the dataset. Additionally, the study is extended to a benchmark dataset of 400 images to investigate how dataset size impacts the performance of these frameworks.

RESULTS

Despite variations in objectives and evaluation metrics between pose estimation and landmark localization tasks, the results are promising. Detectron2's variant outperforms others with an accuracy of 85.89%, compared to 72.92% achieved by YOLOv8's variant on the DiverseCEPH19 dataset. This superior performance is also observed in the smaller benchmark dataset, where Detectron2 consistently maintains higher accuracy than YOLOv8.

CONCLUSION

The noted enhancements in annotation precision suggest the suitability of Detectron2 for deployment in applications that require high precision while taking into account factors such as model size, inference time, and resource utilization, the evidence favors YOLOv8 baselines.

Performance evaluation of a deep learning-based cascaded HRNet model for automatic measurement of X-ray imaging parameters of lumbar sagittal curvature

PURPOSE

To develop a deep learning-based cascaded HRNet model, in order to automatically measure X-ray imaging parameters of lumbar sagittal curvature and to evaluate its prediction performance.

METHODS

A total of 3730 lumbar lateral digital radiography (DR) images were collected from picture archiving and communication system (PACS). Among them, 3150 images were randomly selected as the training dataset and validation dataset, and 580 images as the test dataset. The landmarks of the lumbar curve index (LCI), lumbar lordosis angle (LLA), sacral slope (SS), lumbar lordosis index (LLI), and the posterior edge tangent angle of the vertebral body (PTA) were identified and marked. The measured results of landmarks on the test dataset were compared with the mean values of manual measurement as the reference standard. Percentage of correct key-points (PCK), intra-class correlation coefficient (ICC), Pearson correlation coefficient (r), mean absolute error (MAE), mean square error (MSE), root-mean-square error (RMSE), and Bland-Altman plot were used to evaluate the performance of the cascade HRNet model.

RESULTS

The PCK of the cascaded HRNet model was 97.9-100% in the 3 mm distance threshold. The mean differences between the reference standard and the predicted values for LCI, LLA, SS, LLI, and PTA were 0.43 mm, 0.99°, 1.11°, 0.01 mm, and 0.23°, respectively. There were strong correlation and consistency of the five parameters between the cascaded HRNet model and manual measurements (ICC = 0.989-0.999, R = 0.991-0.999, MAE = 0.63-1.65, MSE = 0.61-4.06, RMSE = 0.78-2.01).

CONCLUSION

The cascaded HRNet model based on deep learning algorithm could accurately identify the sagittal curvature-related landmarks on lateral lumbar DR images and automatically measure the relevant parameters, which is of great significance in clinical application.

Clinical benchmark dataset for AI accuracy analysis: quantifying radiographic annotation of pelvic tilt

Radiographic landmark annotation determines patients' anatomical parameters and influences diagnoses. However, challenges arise from ambiguous region-based definitions, human error, and image quality variations, potentially compromising patient care. Additionally, AI landmark localization often presents its predictions in a probability-based heatmap format, which lacks a corresponding clinical standard for accuracy validation. This Data Descriptor presents a clinical benchmark dataset for pelvic tilt landmarks, gathered through a probabilistic approach to measure annotation accuracy within clinical environments. A retrospective analysis of 115 pelvic sagittal radiographs was conducted for annotating pelvic tilt parameters by five annotators, revealing landmark cloud sizes of 6.04 mm-17.90 mm at a 95% dataset threshold, corresponding to 9.51°-16.55° maximum angular disagreement in clinical settings. The outcome provides a quantified point cloud dataset for each landmark corresponding to different probabilities, which enables assessment of directional annotation distribution and parameter-wise impact, providing clinical benchmarks. The data is readily reusable for AI studies analyzing the same landmarks, and the method can be easily replicated for establishing clinical accuracy benchmarks of other landmarks.

Implicit Is Not Enough: Explicitly Enforcing Anatomical Priors inside Landmark Localization Models

The task of localizing distinct anatomical structures in medical image data is an essential prerequisite for several medical applications, such as treatment planning in orthodontics, bone-age estimation, or initialization of segmentation methods in automated image analysis tools. Currently, Anatomical Landmark Localization (ALL) is mainly solved by deep-learning methods, which cannot guarantee robust ALL predictions; there may always be outlier predictions that are far from their ground truth locations due to out-of-distribution inputs. However, these localization outliers are detrimental to the performance of subsequent medical applications that rely on ALL results. The current ALL literature relies heavily on implicit anatomical constraints built into the loss function and network architecture to reduce the risk of anatomically infeasible predictions. However, we argue that in medical imaging, where images are generally acquired in a controlled environment, we should use stronger explicit anatomical constraints to reduce the number of outliers as much as possible. Therefore, we propose the end-to-end trainable Global Anatomical Feasibility Filter and Analysis (GAFFA) method, which uses prior anatomical knowledge estimated from data to explicitly enforce anatomical constraints. GAFFA refines the initial localization results of a U-Net by approximately solving a Markov Random Field (MRF) with a single iteration of the sum-product algorithm in a differentiable manner. Our experiments demonstrate that GAFFA outperforms all other landmark refinement methods investigated in our framework. Moreover, we show that GAFFA is more robust to large outliers than state-of-the-art methods on the studied X-ray hand dataset. We further motivate this claim by visualizing the anatomical constraints used in GAFFA as spatial energy heatmaps, which allowed us to find an annotation error in the hand dataset not previously discussed in the literature.

Rethinking Self-Training for Semi-Supervised Landmark Detection: A Selection-Free Approach

Self-training is a simple yet effective method for semi-supervised learning, during which pseudo-label selection plays an important role for handling confirmation bias. Despite its popularity, applying self-training to landmark detection faces three problems: 1) The selected confident pseudo-labels often contain data bias, which may hurt model performance; 2) It is not easy to decide a proper threshold for sample selection as the localization task can be sensitive to noisy pseudo-labels; 3) coordinate regression does not output confidence, making selection-based self-training infeasible. To address the above issues, we propose Self-Training for Landmark Detection (STLD), a method that does not require explicit pseudo-label selection. Instead, STLD constructs a task curriculum to deal with confirmation bias, which progressively transitions from more confident to less confident tasks over the rounds of self-training. Pseudo pretraining and shrink regression are two essential components for such a curriculum, where the former is the first task of the curriculum for providing a better model initialization and the latter is further added in the later rounds to directly leverage the pseudo-labels in a coarse-to-fine manner. Experiments on three facial and one medical landmark detection benchmark show that STLD outperforms the existing methods consistently in both semi- and omni-supervised settings. The code is available at https://github.com/jhb86253817/STLD.

Fully Automated Hippocampus Segmentation using T2-informed Deep Convolutional Neural Networks

Hippocampal atrophy (tissue loss) has become a fundamental outcome parameter in clinical trials on Alzheimer's disease. To accurately estimate hippocampus volume and track its volume loss, a robust and reliable segmentation is essential. Manual hippocampus segmentation is considered the gold standard but is extensive, time-consuming, and prone to rater bias. Therefore, it is often replaced by automated programs like FreeSurfer, one of the most commonly used tools in clinical research. Recently, deep learning-based methods have also been successfully applied to hippocampus segmentation. The basis of all approaches are clinically used T1-weighted whole-brain MR images with approximately 1 mm isotropic resolution. However, such T1 images show low contrast-to-noise ratios (CNRs), particularly for many hippocampal substructures, limiting delineation reliability. To overcome these limitations, high-resolution T2-weighted scans are suggested for better visualization and delineation, as they show higher CNRs and usually allow for higher resolutions. Unfortunately, such time-consuming T2-weighted sequences are not feasible in a clinical routine. We propose an automated hippocampus segmentation pipeline leveraging deep learning with T2-weighted MR images for enhanced hippocampus segmentation of clinical T1-weighted images based on a series of 3D convolutional neural networks and a specifically acquired multi-contrast dataset. This dataset consists of corresponding pairs of T1- and high-resolution T2-weighted images, with the T2 images only used to create more accurate manual ground truth annotations and to train the segmentation network. The T2-based ground truth labels were also used to evaluate all experiments by comparing the masks visually and by various quantitative measures. We compared our approach with four established state-of-the-art hippocampus segmentation algorithms (FreeSurfer, ASHS, HippoDeep, HippMapp3r) and demonstrated a superior segmentation performance. Moreover, we found that the automated segmentation of T1-weighted images benefits from the T2-based ground truth data. In conclusion, this work showed the beneficial use of high-resolution, T2-based ground truth data for training an automated, deep learning-based hippocampus segmentation and provides the basis for a reliable estimation of hippocampal atrophy in clinical studies.

Knee landmarks detection via deep learning for automatic imaging evaluation of trochlear dysplasia and patellar height

OBJECTIVES

To develop and validate a deep learning-based approach to automatically measure the patellofemoral instability (PFI) indices related to patellar height and trochlear dysplasia in knee magnetic resonance imaging (MRI) scans.

METHODS

A total of 763 knee MRI slices from 95 patients were included in the study, and 3393 anatomical landmarks were annotated for measuring sulcus angle (SA), trochlear facet asymmetry (TFA), trochlear groove depth (TGD) and lateral trochlear inclination (LTI) to assess trochlear dysplasia, and Insall-Salvati index (ISI), modified Insall-Salvati index (MISI), Caton Deschamps index (CDI) and patellotrochlear index (PTI) to assess patellar height. A U-Net based network was implemented to predict the landmarks' locations. The successful detection rate (SDR) and the mean absolute error (MAE) evaluation metrics were used to evaluate the performance of the network. The intraclass correlation coefficient (ICC) was also used to evaluate the reliability of the proposed framework to measure the mentioned PFI indices.

RESULTS

The developed models achieved good accuracy in predicting the landmarks' locations, with a maximum value for the MAE of 1.38 ± 0.76 mm. The results show that LTI, TGD, ISI, CDI and PTI can be measured with excellent reliability (ICC > 0.9), and SA, TFA and MISI can be measured with good reliability (ICC > 0.75), with the proposed framework.

CONCLUSIONS

This study proposes a reliable approach with promising applicability for automatic patellar height and trochlear dysplasia assessment, assisting the radiologists in their clinical practice.

CLINICAL RELEVANCE STATEMENT

The objective knee landmarks detection on MRI images provided by artificial intelligence may improve the reproducibility and reliability of the imaging evaluation of trochlear anatomy and patellar height, assisting radiologists in their clinical practice in the patellofemoral instability assessment.

KEY POINTS

• Imaging evaluation of patellofemoral instability is subjective and vulnerable to substantial intra and interobserver variability. • Patellar height and trochlear dysplasia are reliably assessed in MRI by means of artificial intelligence (AI). • The developed AI framework provides an objective evaluation of patellar height and trochlear dysplasia enhancing the clinical practice of the radiologists.

pyCEPS: A cross-platform electroanatomic mapping data to computational model conversion platform for the calibration of digital twin models of cardiac electrophysiology

BACKGROUND AND OBJECTIVE

Data from electro-anatomical mapping (EAM) systems are playing an increasingly important role in computational modeling studies for the patient-specific calibration of digital twin models. However, data exported from commercial EAM systems are challenging to access and parse. Converting to data formats that are easily amenable to be viewed and analyzed with commonly used cardiac simulation software tools such as openCARP remains challenging. We therefore developed an open-source platform, pyCEPS, for parsing and converting clinical EAM data conveniently to standard formats widely adopted within the cardiac modeling community.

METHODS AND RESULTS

pyCEPS is an open-source Python-based platform providing the following functions: (i) access and interrogate the EAM data exported from clinical mapping systems; (ii) efficient browsing of EAM data to preview mapping procedures, electrograms (EGMs), and electro-cardiograms (ECGs); (iii) conversion to modeling formats according to the openCARP standard, to be amenable to analysis with standard tools and advanced workflows as used for in silico EAM data. Documentation and training material to facilitate access to this complementary research tool for new users is provided. We describe the technological underpinnings and demonstrate the capabilities of pyCEPS first, and showcase its use in an exemplary modeling application where we use clinical imaging data to build a patient-specific anatomical model.

CONCLUSION

With pyCEPS we offer an open-source framework for accessing EAM data, and converting these to cardiac modeling standard formats. pyCEPS provides the core functionality needed to integrate EAM data in cardiac modeling research. We detail how pyCEPS could be integrated into model calibration workflows facilitating the calibration of a computational model based on EAM data.

Which images to label for few-shot medical image analysis?

The success of deep learning methodologies hinges upon the availability of meticulously labeled extensive datasets. However, when dealing with medical images, the annotation process for such abundant training data often necessitates the involvement of experienced radiologists, thereby consuming their limited time resources. In order to alleviate this burden, few-shot learning approaches have been developed, which manage to achieve competitive performance levels with only several labeled images. Nevertheless, a crucial yet previously overlooked problem in few-shot learning is about the selection of template images for annotation before learning, which affects the final performance. In this study, we propose a novel TEmplate Choosing Policy (TECP) that aims to identify and select "the most worthy" images for annotation, particularly within the context of multiple few-shot medical tasks, including landmark detection, anatomy detection, and anatomy segmentation. TECP is composed of four integral components: (1) Self-supervised training, which entails training a pre-existing deep model to extract salient features from radiological images; (2) Alternative proposals for localizing informative regions within the images; and (3) Representative Score Estimation, which involves the evaluation and identification of the most representative samples or templates. (4) Ranking, which rank all candidates and select one with highest representative score. The efficacy of the TECP approach is demonstrated through a series of comprehensive experiments conducted on multiple public datasets. Across all three medical tasks, the utilization of TECP yields noticeable improvements in model performance.

Automated measurement of lumbar pedicle screw parameters using deep learning algorithm on preoperative CT scans

Purpose

This study aims to devise and assess an automated measurement framework for lumbar pedicle screw parameters leveraging preoperative computed tomography (CT) scans and a deep learning algorithm.

Methods

A deep learning model was constructed employing a dataset comprising 1410 axial preoperative CT images of lumbar pedicles sourced from 282 patients. The model was trained to predict several screw parameters, including the axial angle and width of pedicles, the length of pedicle screw paths, and the interpedicular distance. The mean values of these parameters, as determined by two radiologists and one spinal surgeon, served as the reference standard.

Results

The deep learning model achieved high agreement with the reference standard for the axial angle of the left pedicle (ICC = 0.92) and right pedicle (ICC = 0.93), as well as for the length of the left pedicle screw path (ICC = 0.82) and right pedicle (ICC = 0.87). Similarly, high agreement was observed for pedicle width (left ICC = 0.97, right ICC = 0.98) and interpedicular distance (ICC = 0.91). Overall, the model's performance paralleled that of manual determination of lumbar pedicle screw parameters.

Conclusion

The developed deep learning-based model demonstrates proficiency in accurately identifying landmarks on preoperative CT scans and autonomously generating parameters relevant to lumbar pedicle screw placement. These findings suggest its potential to offer efficient and precise measurements for clinical applications.

Craniomaxillofacial landmarks detection in CT scans with limited labeled data via semi-supervised learning

Background

Three-dimensional cephalometric analysis is crucial in craniomaxillofacial assessment, with landmarks detection in craniomaxillofacial (CMF) CT scans being a key component. However, creating robust deep learning models for this task typically requires extensive CMF CT datasets annotated by experienced medical professionals, a process that is time-consuming and labor-intensive. Conversely, acquiring large volume of unlabeled CMF CT data is relatively straightforward. Thus, semi-supervised learning (SSL), leveraging limited labeled data supplemented by sufficient unlabeled dataset, could be a viable solution to this challenge.

Method

We developed an SSL model, named CephaloMatch, based on a strong-weak perturbation consistency framework. The proposed SSL model incorporates a head position rectification technique through coarse detection to enhance consistency between labeled and unlabeled datasets and a multilayers perturbation method which is employed to expand the perturbation space. The proposed SSL model was assessed using 362 CMF CT scans, divided into a training set (60 scans), a validation set (14 scans), and an unlabeled set (288 scans).

Result

The proposed SSL model attained a detection error of 1.60 ± 0.87 mm, significantly surpassing the performance of conventional fully supervised learning model (1.94 ± 1.12 mm). Notably, the proposed SSL model achieved equivalent detection accuracy (1.91 ± 1.00 mm) with only half the labeled dataset, compared to the fully supervised learning model.

Conclusions

The proposed SSL model demonstrated exceptional performance in landmarks detection using a limited labeled CMF CT dataset, significantly reducing the workload of medical professionals and enhances the accuracy of 3D cephalometric analysis.

Landmark Localization From Medical Images With Generative Distribution Prior

In medical image analysis, anatomical landmarks usually contain strong prior knowledge of their structural information. In this paper, we propose to promote medical landmark localization by modeling the underlying landmark distribution via normalizing flows. Specifically, we introduce the flow-based landmark distribution prior as a learnable objective function into a regression-based landmark localization framework. Moreover, we employ an integral operation to make the mapping from heatmaps to coordinates differentiable to further enhance heatmap-based localization with the learned distribution prior. Our proposed Normalizing Flow-based Distribution Prior (NFDP) employs a straightforward backbone and non-problem-tailored architecture (i.e., ResNet18), which delivers high-fidelity outputs across three X-ray-based landmark localization datasets. Remarkably, the proposed NFDP can do the job with minimal additional computational burden as the normalizing flows module is detached from the framework on inferencing. As compared to existing techniques, our proposed NFDP provides a superior balance between prediction accuracy and inference speed, making it a highly efficient and effective approach. The source code of this paper is available at https://github.com/jacksonhzx95/NFDP.

A comprehensive survey on deep active learning in medical image analysis

Deep learning has achieved widespread success in medical image analysis, leading to an increasing demand for large-scale expert-annotated medical image datasets. Yet, the high cost of annotating medical images severely hampers the development of deep learning in this field. To reduce annotation costs, active learning aims to select the most informative samples for annotation and train high-performance models with as few labeled samples as possible. In this survey, we review the core methods of active learning, including the evaluation of informativeness and sampling strategy. For the first time, we provide a detailed summary of the integration of active learning with other label-efficient techniques, such as semi-supervised, self-supervised learning, and so on. We also summarize active learning works that are specifically tailored to medical image analysis. Additionally, we conduct a thorough comparative analysis of the performance of different AL methods in medical image analysis with experiments. In the end, we offer our perspectives on the future trends and challenges of active learning and its applications in medical image analysis. An accompanying paper list and code for the comparative analysis is available in https://github.com/LightersWang/Awesome-Active-Learning-for-Medical-Image-Analysis.

Weakly supervised deep learning for diagnosis of multiple vertebral compression fractures in CT

OBJECTIVE

This study aims to develop a weakly supervised deep learning (DL) model for vertebral-level vertebral compression fracture (VCF) classification using image-level labelled data.

METHODS

The training set included 815 patients with normal (n = 507, 62%) or VCFs (n = 308, 38%). Our proposed model was trained on image-level labelled data for vertebral-level classification. Another supervised DL model was trained with vertebral-level labelled data to compare the performance of the proposed model.

RESULTS

The test set included 227 patients with normal (n = 117, 52%) or VCFs (n = 110, 48%). For a fair comparison of the two models, we compared sensitivities with the same specificities of the proposed model and the vertebral-level supervised model. The specificity for overall L1-L5 performance was 0.981. The proposed model may outperform the vertebral-level supervised model with sensitivities of 0.770 vs 0.705 (p = 0.080), respectively. For vertebral-level analysis, the specificities for each L1-L5 were 0.974, 0.973, 0.970, 0.991, and 0.995, respectively. The proposed model yielded the same or better sensitivity than the vertebral-level supervised model in L1 (0.750 vs 0.694, p = 0.480), L3 (0.793 vs 0.586, p < 0.05), L4 (0.833 vs 0.667, p = 0.480), and L5 (0.600 vs 0.600, p = 1.000), respectively. The proposed model showed lower sensitivity than the vertebral-level supervised model for L2, but there was no significant difference (0.775 vs 0.825, p = 0.617).

CONCLUSIONS

The proposed model may have a comparable or better performance than the supervised model in vertebral-level VCF classification.

CLINICAL RELEVANCE STATEMENT

Vertebral-level vertebral compression fracture classification aids in devising patient-specific treatment plans by identifying the precise vertebrae affected by compression fractures.

KEY POINTS

• Our proposed weakly supervised method may have comparable or better performance than the supervised method for vertebral-level vertebral compression fracture classification. • The weakly supervised model could have classified cases with multiple vertebral compression fractures at the vertebral-level, even if the model was trained with image-level labels. • Our proposed method could help reduce radiologists' labour because it enables vertebral-level classification from image-level labels.

Multi-task global optimization-based method for vascular landmark detection

Vascular landmark detection plays an important role in medical analysis and clinical treatment. However, due to the complex topology and similar local appearance around landmarks, the popular heatmap regression based methods always suffer from the landmark confusion problem. Vascular landmarks are connected by vascular segments and have special spatial correlations, which can be utilized for performance improvement. In this paper, we propose a multi-task global optimization-based framework for accurate and automatic vascular landmark detection. A multi-task deep learning network is exploited to accomplish landmark heatmap regression, vascular semantic segmentation, and orientation field regression simultaneously. The two auxiliary objectives are highly correlated with the heatmap regression task and help the network incorporate the structural prior knowledge. During inference, instead of performing a max-voting strategy, we propose a global optimization-based post-processing method for final landmark decision. The spatial relationships between neighboring landmarks are utilized explicitly to tackle the landmark confusion problem. We evaluated our method on a cerebral MRA dataset with 564 volumes, a cerebral CTA dataset with 510 volumes, and an aorta CTA dataset with 50 volumes. The experiments demonstrate that the proposed method is effective for vascular landmark localization and achieves state-of-the-art performance.

The Future of Orthodontics: Deep Learning Technologies

Deep learning has emerged as a revolutionary technical advancement in modern orthodontics, offering novel methods for diagnosis, treatment planning, and outcome prediction. Over the past 25 years, the field of dentistry has widely adopted information technology (IT), resulting in several benefits, including decreased expenses, increased efficiency, decreased need for human expertise, and reduced errors. The transition from preset rules to learning from real-world examples, particularly machine learning (ML) and artificial intelligence (AI), has greatly benefited the organization, analysis, and storage of medical data. Deep learning, a type of AI, enables robots to mimic human neural networks, allowing them to learn and make decisions independently without the need for explicit programming. Its ability to automate cephalometric analysis and enhance diagnosis through 3D imaging has revolutionized orthodontic operations. Deep learning models have the potential to significantly improve treatment outcomes and reduce human errors by accurately identifying anatomical characteristics on radiographs, thereby expediting analytical processes. Additionally, the use of 3D imaging technologies such as cone-beam computed tomography (CBCT) can facilitate precise treatment planning, allowing for comprehensive examinations of craniofacial architecture, tooth movements, and airway dimensions. In today's era of personalized medicine, deep learning's ability to customize treatments for individual patients has propelled the field of orthodontics forward tremendously. However, it is essential to address issues related to data privacy, model interpretability, and ethical considerations before orthodontic practices can use deep learning in an ethical and responsible manner. Modern orthodontics is evolving, thanks to the ability of deep learning to deliver more accurate, effective, and personalized orthodontic treatments, improving patient care as technology develops.

Knowledge distillation on individual vertebrae segmentation exploiting 3D U-Net

Recent advances in medical imaging have highlighted the critical development of algorithms for individual vertebral segmentation on computed tomography (CT) scans. Essential for diagnostic accuracy and treatment planning in orthopaedics, neurosurgery and oncology, these algorithms face challenges in clinical implementation, including integration into healthcare systems. Consequently, our focus lies in exploring the application of knowledge distillation (KD) methods to train shallower networks capable of efficiently segmenting vertebrae in CT scans. This approach aims to reduce segmentation time, enhance suitability for emergency cases, and optimize computational and memory resource efficiency. Building upon prior research in the field, a two-step segmentation approach was employed. Firstly, the spine's location was determined by predicting a heatmap, indicating the probability of each voxel belonging to the spine. Subsequently, an iterative segmentation of vertebrae was performed from the top to the bottom of the CT volume over the located spine, using a memory instance to record the already segmented vertebrae. KD methods were implemented by training a teacher network with performance similar to that found in the literature, and this knowledge was distilled to a shallower network (student). Two KD methods were applied: (1) using the soft outputs of both networks and (2) matching logits. Two publicly available datasets, comprising 319 CT scans from 300 patients and a total of 611 cervical, 2387 thoracic, and 1507 lumbar vertebrae, were used. To ensure dataset balance and robustness, effective data augmentation methods were applied, including cleaning the memory instance to replicate the first vertebra segmentation. The teacher network achieved an average Dice similarity coefficient (DSC) of 88.22% and a Hausdorff distance (HD) of 7.71 mm, showcasing performance similar to other approaches in the literature. Through knowledge distillation from the teacher network, the student network's performance improved, with an average DSC increasing from 75.78% to 84.70% and an HD decreasing from 15.17 mm to 8.08 mm. Compared to other methods, our teacher network exhibited up to 99.09% fewer parameters, 90.02% faster inference time, 88.46% shorter total segmentation time, and 89.36% less associated carbon (CO2) emission rate. Regarding our student network, it featured 75.00% fewer parameters than our teacher, resulting in a 36.15% reduction in inference time, a 33.33% decrease in total segmentation time, and a 42.96% reduction in CO2 emissions. This study marks the first exploration of applying KD to the problem of individual vertebrae segmentation in CT, demonstrating the feasibility of achieving comparable performance to existing methods using smaller neural networks.

Landmark annotation through feature combinations: a comparative study on cephalometric images with in-depth analysis of model's explainability

OBJECTIVES

The objectives of this study are to explore and evaluate the automation of anatomical landmark localization in cephalometric images using machine learning techniques, with a focus on feature extraction and combinations, contextual analysis, and model interpretability through Shapley Additive exPlanations (SHAP) values.

METHODS

We conducted extensive experimentation on a private dataset of 300 lateral cephalograms to thoroughly study the annotation results obtained using pixel feature descriptors including raw pixel, gradient magnitude, gradient direction, and histogram-oriented gradient (HOG) values. The study includes evaluation and comparison of these feature descriptions calculated at different contexts namely local, pyramid, and global. The feature descriptor obtained using individual combinations is used to discern between landmark and nonlandmark pixels using classification method. Additionally, this study addresses the opacity of LGBM ensemble tree models across landmarks, introducing SHAP values to enhance interpretability.

RESULTS

The performance of feature combinations was assessed using metrics like mean radial error, standard deviation, success detection rate (SDR) (2 mm), and test time. Remarkably, among all the combinations explored, both the HOG and gradient direction operations demonstrated significant performance across all context combinations. At the contextual level, the global texture outperformed the others, although it came with the trade-off of increased test time. The HOG in the local context emerged as the top performer with an SDR of 75.84% compared to others.

CONCLUSIONS

The presented analysis enhances the understanding of the significance of different features and their combinations in the realm of landmark annotation but also paves the way for further exploration of landmark-specific feature combination methods, facilitated by explainability.

An Anatomy- and Topology-Preserving Framework for Coronary Artery Segmentation

Coronary artery segmentation is critical for coronary artery disease diagnosis but challenging due to its tortuous course with numerous small branches and inter-subject variations. Most existing studies ignore important anatomical information and vascular topologies, leading to less desirable segmentation performance that usually cannot satisfy clinical demands. To deal with these challenges, in this paper we propose an anatomy- and topology-preserving two-stage framework for coronary artery segmentation. The proposed framework consists of an anatomical dependency encoding (ADE) module and a hierarchical topology learning (HTL) module for coarse-to-fine segmentation, respectively. Specifically, the ADE module segments four heart chambers and aorta, and thus five distance field maps are obtained to encode distance between chamber surfaces and coarsely segmented coronary artery. Meanwhile, ADE also performs coronary artery detection to crop region-of-interest and eliminate foreground-background imbalance. The follow-up HTL module performs fine segmentation by exploiting three hierarchical vascular topologies, i.e., key points, centerlines, and neighbor connectivity using a multi-task learning scheme. In addition, we adopt a bottom-up attention interaction (BAI) module to integrate the feature representations extracted across hierarchical topologies. Extensive experiments on public and in-house datasets show that the proposed framework achieves state-of-the-art performance for coronary artery segmentation.

Automatic detection of posterior superior alveolar artery in dental cone-beam CT images using a deeply supervised multi-scale 3D network

OBJECTIVES

This study aimed to develop a robust and accurate deep learning network for detecting the posterior superior alveolar artery (PSAA) in dental cone-beam CT (CBCT) images, focusing on the precise localization of the centre pixel as a critical centreline pixel.

METHODS

PSAA locations were manually labelled on dental CBCT data from 150 subjects. The left maxillary sinus images were horizontally flipped. In total, 300 datasets were created. Six different deep learning networks were trained, including 3D U-Net, deeply supervised 3D U-Net (3D U-Net DS), multi-scale deeply supervised 3D U-Net (3D U-Net MSDS), 3D Attention U-Net, 3D V-Net, and 3D Dense U-Net. The performance evaluation involved predicting the centre pixel of the PSAA. This was assessed using mean absolute error (MAE), mean radial error (MRE), and successful detection rate (SDR).

RESULTS

The 3D U-Net MSDS achieved the best prediction performance among the tested networks, with an MAE measurement of 0.696 ± 1.552 mm and MRE of 1.101 ± 2.270 mm. In comparison, the 3D U-Net showed the lowest performance. The 3D U-Net MSDS demonstrated a SDR of 95% within a 2 mm MAE. This was a significantly higher result than other networks that achieved a detection rate of over 80%.

CONCLUSIONS

This study presents a robust deep learning network for accurate PSAA detection in dental CBCT images, emphasizing precise centre pixel localization. The method achieves high accuracy in locating small vessels, such as the PSAA, and has the potential to enhance detection accuracy and efficiency, thus impacting oral and maxillofacial surgery planning and decision-making.

Two-stage multi-task deep learning framework for simultaneous pelvic bone segmentation and landmark detection from CT images

PURPOSE

Pelvic bone segmentation and landmark definition from computed tomography (CT) images are prerequisite steps for the preoperative planning of total hip arthroplasty. In clinical applications, the diseased pelvic anatomy usually degrades the accuracies of bone segmentation and landmark detection, leading to improper surgery planning and potential operative complications.

METHODS

This work proposes a two-stage multi-task algorithm to improve the accuracy of pelvic bone segmentation and landmark detection, especially for the diseased cases. The two-stage framework uses a coarse-to-fine strategy which first conducts global-scale bone segmentation and landmark detection and then focuses on the important local region to further refine the accuracy. For the global stage, a dual-task network is designed to share the common features between the segmentation and detection tasks, so that the two tasks mutually reinforce each other's performance. For the local-scale segmentation, an edge-enhanced dual-task network is designed for simultaneous bone segmentation and edge detection, leading to the more accurate delineation of the acetabulum boundary.

RESULTS

This method was evaluated via threefold cross-validation based on 81 CT images (including 31 diseased and 50 healthy cases). The first stage achieved DSC scores of 0.94, 0.97, and 0.97 for the sacrum, left and right hips, respectively, and an average distance error of 3.24 mm for the bone landmarks. The second stage further improved the DSC of the acetabulum by 5.42%, and this accuracy outperforms the state-of-the-arts (SOTA) methods by 0.63%. Our method also accurately segmented the diseased acetabulum boundaries. The entire workflow took ~ 10 s, which was only half of the U-Net run time.

CONCLUSION

Using the multi-task networks and the coarse-to-fine strategy, this method achieved more accurate bone segmentation and landmark detection than the SOTA method, especially for diseased hip images. Our work contributes to accurate and rapid design of acetabular cup prostheses.

Evaluating a Periapical Lesion Detection CNN on a Clinically Representative CBCT Dataset-A Validation Study

The aim of this validation study was to comprehensively evaluate the performance and generalization capability of a deep learning-based periapical lesion detection algorithm on a clinically representative cone-beam computed tomography (CBCT) dataset and test for non-inferiority. The evaluation involved 195 CBCT images of adult upper and lower jaws, where sensitivity and specificity metrics were calculated for all teeth, stratified by jaw, and stratified by tooth type. Furthermore, each lesion was assigned a periapical index score based on its size to enable a score-based evaluation. Non-inferiority tests were conducted with proportions of 90% for sensitivity and 82% for specificity. The algorithm achieved an overall sensitivity of 86.7% and a specificity of 84.3%. The non-inferiority test indicated the rejection of the null hypothesis for specificity but not for sensitivity. However, when excluding lesions with a periapical index score of one (i.e., very small lesions), the sensitivity improved to 90.4%. Despite the challenges posed by the dataset, the algorithm demonstrated promising results. Nevertheless, further improvements are needed to enhance the algorithm's robustness, particularly in detecting very small lesions and the handling of artifacts and outliers commonly encountered in real-world clinical scenarios.

An automated optimization pipeline for clinical-grade computer-assisted planning of high tibial osteotomies under consideration of weight-bearing

3D preoperative planning for high tibial osteotomies (HTO) has increasingly replaced 2D planning but is complex, time-consuming and therefore expensive. Several interdependent clinical objectives and constraints have to be considered, which often requires multiple rounds of revisions between surgeons and biomedical engineers. We therefore developed an automated preoperative planning pipeline, which takes imaging data as an input to generate a ready-to-use, patient-specific planning solution. Deep-learning based segmentation and landmark localization was used to enable the fully automated 3D lower limb deformity assessment. A 2D-3D registration algorithm allowed the transformation of the 3D bone models into the weight-bearing state. Finally, an optimization framework was implemented to generate ready-to use preoperative plannings in a fully automated fashion, using a genetic algorithm to solve the multi-objective optimization (MOO) problem based on several clinical requirements and constraints. The entire pipeline was evaluated on a large clinical dataset of 53 patient cases who previously underwent a medial opening-wedge HTO. The pipeline was used to automatically generate preoperative solutions for these patients. Five experts blindly compared the automatically generated solutions to the previously generated manual plannings. The overall mean rating for the algorithm-generated solutions was better than for the manual solutions. In 90% of all comparisons, they were considered to be equally good or better than the manual solution. The combined use of deep learning approaches, registration methods and MOO can reliably produce ready-to-use preoperative solutions that significantly reduce human workload and related health costs.

Towards Improved Identification of Vertebral Fractures in Routine Computed Tomography (CT) Scans: Development and External Validation of a Machine Learning Algorithm

Vertebral fractures (VFs) are the hallmark of osteoporosis, being one of the most frequent types of fragility fracture and an early sign of the disease. They are associated with significant morbidity and mortality. VFs are incidentally found in one out of five imaging studies, however, more than half of the VFs are not identified nor reported in patient computed tomography (CT) scans. Our study aimed to develop a machine learning algorithm to identify VFs in abdominal/chest CT scans and evaluate its performance. We acquired two independent data sets of routine abdominal/chest CT scans of patients aged 50 years or older: a training set of 1011 scans from a non-interventional, prospective proof-of-concept study at the Universitair Ziekenhuis (UZ) Brussel and a validation set of 2000 subjects from an observational cohort study at the Hospital of Holbaek. Both data sets were externally reevaluated to identify reference standard VF readings using the Genant semiquantitative (SQ) grading. Four independent models have been trained in a cross-validation experiment using the training set and an ensemble of four models has been applied to the external validation set. The validation set contained 15.3% scans with one or more VF (SQ2-3), whereas 663 of 24,930 evaluable vertebrae (2.7%) were fractured (SQ2-3) as per reference standard readings. Comparison of the ensemble model with the reference standard readings in identifying subjects with one or more moderate or severe VF resulted in an area under the receiver operating characteristic curve (AUROC) of 0.88 (95% confidence interval [CI], 0.85-0.90), accuracy of 0.92 (95% CI, 0.91-0.93), kappa of 0.72 (95% CI, 0.67-0.76), sensitivity of 0.81 (95% CI, 0.76-0.85), and specificity of 0.95 (95% CI, 0.93-0.96). We demonstrated that a machine learning algorithm trained for VF detection achieved strong performance on an external validation set. It has the potential to support healthcare professionals with the early identification of VFs and prevention of future fragility fractures. © 2023 UCB S.A. and The Authors. Journal of Bone and Mineral Research published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research (ASBMR).

Automated computation of radiographic parameters of distal radial metaphyseal fractures in forearm X-rays

PURPOSE

Radiographic parameters (RPs) provide objective support for effective decision making in determining clinical treatment of distal radius fractures (DRFs). This paper presents a novel automatic RP computation pipeline for computing the six anatomical RPs associated with DRFs in anteroposterior (AP) and lateral (LAT) forearm radiographs.

METHODS

The pipeline consists of: (1) segmentation of the distal radius and ulna bones with six 2D Dynamic U-Net deep learning models; (2) landmark points detection and distal radius axis computation from the segmentations with geometric methods; (3) RP computation and generation of a quantitative DRF report and composite AP and LAT radiograph images. This hybrid approach combines the advantages of deep learning and model-based methods.

RESULTS

The pipeline was evaluated on 90 AP and 93 LAT radiographs for which ground truth distal radius and ulna segmentations and RP landmarks were manually obtained by expert clinicians. It achieves an accuracy of 94 and 86% on the AP and LAT RPs, within the observer variability, and an RP measurement difference of 1.4 ± 1.2° for the radial angle, 0.5 ± 0.6 mm for the radial length, 0.9 ± 0.7 mm for the radial shift, 0.7 ± 0.5 mm for the ulnar variance, 2.9 ± 3.3° for the palmar tilt and 1.2 ± 1.0 mm for the dorsal shift.

CONCLUSION

Our pipeline is the first fully automatic method that accurately and robustly computes the RPs for a wide variety of clinical forearm radiographs from different sources, hand orientations, with and without cast. The computed accurate and reliable RF measurements may support fracture severity assessment and clinical management.

Automatic craniomaxillofacial landmarks detection in CT images of individuals with dentomaxillofacial deformities by a two-stage deep learning model

BACKGROUND

Accurate cephalometric analysis plays a vital role in the diagnosis and subsequent surgical planning in orthognathic and orthodontics treatment. However, manual digitization of anatomical landmarks in computed tomography (CT) is subject to limitations such as low accuracy, poor repeatability and excessive time consumption. Furthermore, the detection of landmarks has more difficulties on individuals with dentomaxillofacial deformities than normal individuals. Therefore, this study aims to develop a deep learning model to automatically detect landmarks in CT images of patients with dentomaxillofacial deformities.

METHODS

Craniomaxillofacial (CMF) CT data of 80 patients with dentomaxillofacial deformities were collected for model development. 77 anatomical landmarks digitized by experienced CMF surgeons in each CT image were set as the ground truth. 3D UX-Net, the cutting-edge medical image segmentation network, was adopted as the backbone of model architecture. Moreover, a new region division pattern for CMF structures was designed as a training strategy to optimize the utilization of computational resources and image resolution. To evaluate the performance of this model, several experiments were conducted to make comparison between the model and manual digitization approach.

RESULTS

The training set and the validation set included 58 and 22 samples respectively. The developed model can accurately detect 77 landmarks on bone, soft tissue and teeth with a mean error of 1.81 ± 0.89 mm. Removal of region division before training significantly increased the error of prediction (2.34 ± 1.01 mm). In terms of manual digitization, the inter-observer and intra-observer variations were 1.27 ± 0.70 mm and 1.01 ± 0.74 mm respectively. In all divided regions except Teeth Region (TR), our model demonstrated equivalent performance to experienced CMF surgeons in landmarks detection (p  >  0.05).

CONCLUSIONS

The developed model demonstrated excellent performance in detecting craniomaxillofacial landmarks when considering manual digitization work of expertise as benchmark. It is also verified that the region division pattern designed in this study remarkably improved the detection accuracy.

Automated Cephalometric Landmark Detection Using Deep Reinforcement Learning

Accurate cephalometric landmark detection leads to accurate analysis, diagnosis, and surgical planning. Many studies on automated landmark detection have been conducted, however reinforcement learning-based networks have not yet been applied. This is the first study to apply deep Q-network (DQN) and double deep Q-network (DDQN) to automated cephalometric landmark detection to the best of our knowledge. The performance of the DQN-based network for cephalometric landmark detection was evaluated using the IEEE International Symposium of Biomedical Imaging (ISBI) 2015 Challenge data set and compared with the previously proposed methods. Furthermore, the clinical applicability of DQN-based automated cephalometric landmark detection was confirmed by testing the DQN-based and DDQN-based network using 500-patient data collected in a clinic. The DQN-based network demonstrated that the average mean radius error of 19 landmarks was smaller than 2 mm, that is, the clinically accepted level, without data augmentation and additional preprocessing. Our DQN-based and DDQN-based approaches tested with the 500-patient data set showed the average success detection rate of 67.33% and 66.04% accuracy within 2 mm, respectively, indicating the feasibility and potential of clinical application.

Ceph-Net: automatic detection of cephalometric landmarks on scanned lateral cephalograms from children and adolescents using an attention-based stacked regression network

BACKGROUND

The success of cephalometric analysis depends on the accurate detection of cephalometric landmarks on scanned lateral cephalograms. However, manual cephalometric analysis is time-consuming and can cause inter- and intra-observer variability. The purpose of this study was to automatically detect cephalometric landmarks on scanned lateral cephalograms with low contrast and resolution using an attention-based stacked regression network (Ceph-Net).

METHODS

The main body of Ceph-Net compromised stacked fully convolutional networks (FCN) which progressively refined the detection of cephalometric landmarks on each FCN. By embedding dual attention and multi-path convolution modules in Ceph-Net, the network learned local and global context and semantic relationships between cephalometric landmarks. Additionally, the intermediate deep supervision in each FCN further boosted the training stability and the detection performance of cephalometric landmarks.

RESULTS

Ceph-Net showed a superior detection performance in mean radial error and successful detection rate, including accuracy improvements in cephalometric landmark detection located in low-contrast soft tissues compared with other detection networks. Moreover, Ceph-Net presented superior detection performance on the test dataset split by age from 8 to 16 years old.

CONCLUSIONS

Ceph-Net demonstrated an automatic and superior detection of cephalometric landmarks by successfully learning local and global context and semantic relationships between cephalometric landmarks in scanned lateral cephalograms with low contrast and resolutions.

Application of Artificial Intelligence in Orthodontics: Current State and Future Perspectives

In recent years, there has been the notable emergency of artificial intelligence (AI) as a transformative force in multiple domains, including orthodontics. This review aims to provide a comprehensive overview of the present state of AI applications in orthodontics, which can be categorized into the following domains: (1) diagnosis, including cephalometric analysis, dental analysis, facial analysis, skeletal-maturation-stage determination and upper-airway obstruction assessment; (2) treatment planning, including decision making for extractions and orthognathic surgery, and treatment outcome prediction; and (3) clinical practice, including practice guidance, remote care, and clinical documentation. We have witnessed a broadening of the application of AI in orthodontics, accompanied by advancements in its performance. Additionally, this review outlines the existing limitations within the field and offers future perspectives.

Joint Cranial Bone Labeling and Landmark Detection in Pediatric CT Images Using Context Encoding

Image segmentation, labeling, and landmark detection are essential tasks for pediatric craniofacial evaluation. Although deep neural networks have been recently adopted to segment cranial bones and locate cranial landmarks from computed tomography (CT) or magnetic resonance (MR) images, they may be hard to train and provide suboptimal results in some applications. First, they seldom leverage global contextual information that can improve object detection performance. Second, most methods rely on multi-stage algorithm designs that are inefficient and prone to error accumulation. Third, existing methods often target simple segmentation tasks and have shown low reliability in more challenging scenarios such as multiple cranial bone labeling in highly variable pediatric datasets. In this paper, we present a novel end-to-end neural network architecture based on DenseNet that incorporates context regularization to jointly label cranial bone plates and detect cranial base landmarks from CT images. Specifically, we designed a context-encoding module that encodes global context information as landmark displacement vector maps and uses it to guide feature learning for both bone labeling and landmark identification. We evaluated our model on a highly diverse pediatric CT image dataset of 274 normative subjects and 239 patients with craniosynostosis (age 0.63 ± 0.54 years, range 0-2 years). Our experiments demonstrate improved performance compared to state-of-the-art approaches.

Automated cross-sectional view selection in CT angiography of aortic dissections with uncertainty awareness and retrospective clinical annotations

Surveillance imaging of patients with chronic aortic diseases, such as aneurysms and dissections, relies on obtaining and comparing cross-sectional diameter measurements along the aorta at predefined aortic landmarks, over time. The orientation of the cross-sectional measuring planes at each landmark is currently defined manually by highly trained operators. Centerline-based approaches are unreliable in patients with chronic aortic dissection, because of the asymmetric flow channels, differences in contrast opacification, and presence of mural thrombus, making centerline computations or measurements difficult to generate and reproduce. In this work, we present three alternative approaches - INS, MCDS, MCDbS - based on convolutional neural networks and uncertainty quantification methods to predict the orientation (ϕ,θ) of such cross-sectional planes. For the monitoring of chronic aortic dissections, we show how a dataset of 162 CTA volumes with overall 3273 imperfect manual annotations routinely collected in a clinic can be efficiently used to accomplish this task, despite the presence of non-negligible interoperator variabilities in terms of mean absolute error (MAE) and 95% limits of agreement (LOA). We show how, despite the large limits of agreement in the training data, the trained model provides faster and more reproducible results than either an expert user or a centerline method. The remaining disagreement lies within the variability produced by three independent expert annotators and matches the current state of the art, providing a similar error, but in a fraction of the time.

Fully automated pipeline for measurement of the thoracic aorta using joint segmentation and localization neural network

Purpose

Diagnosis and surveillance of thoracic aortic aneurysm (TAA) involves measuring the aortic diameter at various locations along the length of the aorta, often using computed tomography angiography (CTA). Currently, measurements are performed by human raters using specialized software for three-dimensional analysis, a time-consuming process, requiring 15 to 45 min of focused effort. Thus, we aimed to develop a convolutional neural network (CNN)-based algorithm for fully automated and accurate aortic measurements.

Approach

Using 212 CTA scans, we trained a CNN to perform segmentation and localization of key landmarks jointly. Segmentation mask and landmarks are subsequently used to obtain the centerline and cross-sectional diameters of the aorta. Subsequently, a cubic spline is fit to the aortic boundary at the sinuses of Valsalva to avoid errors related inclusions of coronary artery origins. Performance was evaluated on a test set of 60 scans with automated measurements compared against expert manual raters.

Result

Compared to training separate networks for each task, joint training yielded higher accuracy for segmentation, especially at the boundary (p < 0.001 ), but a marginally worse (0.2 to 0.5 mm) accuracy for landmark localization (p < 0.001 ). Mean absolute error between human and automated was ≤ 1    mm at six of nine standard clinical measurement locations. However, higher errors were noted in the aortic root and arch regions, ranging between 1.4 and 2.2 mm, although agreement of manual raters was also lower in these regions.

Conclusion

Fully automated aortic diameter measurements in TAA are feasible using a CNN-based algorithm. Automated measurements demonstrated low errors that are comparable in magnitude to those with manual raters; however, measurement error was highest in the aortic root and arch.

Three-Dimensional Craniofacial Landmark Detection in Series of CT Slices Using Multi-Phased Regression Networks

Geometrical assessments of human skulls have been conducted based on anatomical landmarks. If developed, the automatic detection of these landmarks will yield both medical and anthropological benefits. In this study, an automated system with multi-phased deep learning networks was developed to predict the three-dimensional coordinate values of craniofacial landmarks. Computed tomography images of the craniofacial area were obtained from a publicly available database. They were digitally reconstructed into three-dimensional objects. Sixteen anatomical landmarks were plotted on each of the objects, and their coordinate values were recorded. Three-phased regression deep learning networks were trained using ninety training datasets. For the evaluation, 30 testing datasets were employed. The 3D error for the first phase, which tested 30 data, was 11.60 px on average (1 px = 500/512 mm). For the second phase, it was significantly improved to 4.66 px. For the third phase, it was further significantly reduced to 2.88. This was comparable to the gaps between the landmarks, as plotted by two experienced practitioners. Our proposed method of multi-phased prediction, which conducts coarse detection first and narrows down the detection area, may be a possible solution to prediction problems, taking into account the physical limitations of memory and computation.

Artificial Intelligence for Detecting Cephalometric Landmarks: A Systematic Review and Meta-analysis

Using computer vision through artificial intelligence (AI) is one of the main technological advances in dentistry. However, the existing literature on the practical application of AI for detecting cephalometric landmarks of orthodontic interest in digital images is heterogeneous, and there is no consensus regarding accuracy and precision. Thus, this review evaluated the use of artificial intelligence for detecting cephalometric landmarks in digital imaging examinations and compared it to manual annotation of landmarks. An electronic search was performed in nine databases to find studies that analyzed the detection of cephalometric landmarks in digital imaging examinations with AI and manual landmarking. Two reviewers selected the studies, extracted the data, and assessed the risk of bias using QUADAS-2. Random-effects meta-analyses determined the agreement and precision of AI compared to manual detection at a 95% confidence interval. The electronic search located 7410 studies, of which 40 were included. Only three studies presented a low risk of bias for all domains evaluated. The meta-analysis showed AI agreement rates of 79% (95% CI: 76-82%, I2 = 99%) and 90% (95% CI: 87-92%, I2 = 99%) for the thresholds of 2 and 3 mm, respectively, with a mean divergence of 2.05 (95% CI: 1.41-2.69, I2 = 10%) compared to manual landmarking. The menton cephalometric landmark showed the lowest divergence between both methods (SMD, 1.17; 95% CI, 0.82; 1.53; I2 = 0%). Based on very low certainty of evidence, the application of AI was promising for automatically detecting cephalometric landmarks, but further studies should focus on testing its strength and validity in different samples.

Lumbar spine segmentation method based on deep learning

Aiming at the difficulties of lumbar vertebrae segmentation in computed tomography (CT) images, we propose an automatic lumbar vertebrae segmentation method based on deep learning. The method mainly includes two parts: lumbar vertebra positioning and lumbar vertebrae segmentation. First of all, we propose a lumbar spine localization network of Unet network, which can directly locate the lumbar spine part in the image. Then, we propose a three-dimensional XUnet lumbar vertebrae segmentation method to achieve automatic lumbar vertebrae segmentation. The method proposed in this paper was validated on the lumbar spine CT images on the public dataset VerSe 2020 and our hospital dataset. Through qualitative comparison and quantitative analysis, the experimental results show that the method proposed in this paper can obtain good lumbar vertebrae segmentation performance, which can be further applied to detection of spinal anomalies and surgical treatment.

Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives

Transformer, one of the latest technological advances of deep learning, has gained prevalence in natural language processing or computer vision. Since medical imaging bear some resemblance to computer vision, it is natural to inquire about the status quo of Transformers in medical imaging and ask the question: can the Transformer models transform medical imaging? In this paper, we attempt to make a response to the inquiry. After a brief introduction of the fundamentals of Transformers, especially in comparison with convolutional neural networks (CNNs), and highlighting key defining properties that characterize the Transformers, we offer a comprehensive review of the state-of-the-art Transformer-based approaches for medical imaging and exhibit current research progresses made in the areas of medical image segmentation, recognition, detection, registration, reconstruction, enhancement, etc. In particular, what distinguishes our review lies in its organization based on the Transformer's key defining properties, which are mostly derived from comparing the Transformer and CNN, and its type of architecture, which specifies the manner in which the Transformer and CNN are combined, all helping the readers to best understand the rationale behind the reviewed approaches. We conclude with discussions of future perspectives.