A multi-scale data fusion framework for bone age assessment with convolutional neural networks

BAE-ViT: An Efficient Multimodal Vision Transformer for Bone Age Estimation

This research introduces BAE-ViT, a specialized vision transformer model developed for bone age estimation (BAE). This model is designed to efficiently merge image and sex data, a capability not present in traditional convolutional neural networks (CNNs). BAE-ViT employs a novel data fusion method to facilitate detailed interactions between visual and non-visual data by tokenizing non-visual information and concatenating all tokens (visual or non-visual) as the input to the model. The model underwent training on a large-scale dataset from the 2017 RSNA Pediatric Bone Age Machine Learning Challenge, where it exhibited commendable performance, particularly excelling in handling image distortions compared to existing models. The effectiveness of BAE-ViT was further affirmed through statistical analysis, demonstrating a strong correlation with the actual ground-truth labels. This study contributes to the field by showcasing the potential of vision transformers as a viable option for integrating multimodal data in medical imaging applications, specifically emphasizing their capacity to incorporate non-visual elements like sex information into the framework. This tokenization method not only demonstrates superior performance in this specific task but also offers a versatile framework for integrating multimodal data in medical imaging applications.

Deep learning-based automated bone age estimation for Saudi patients on hand radiograph images: a retrospective study

PURPOSE

In pediatric medicine, precise estimation of bone age is essential for skeletal maturity evaluation, growth disorder diagnosis, and therapeutic intervention planning. Conventional techniques for determining bone age depend on radiologists' subjective judgments, which may lead to non-negligible differences in the estimated bone age. This study proposes a deep learning-based model utilizing a fully connected convolutional neural network(CNN) to predict bone age from left-hand radiographs.

METHODS

The data set used in this study, consisting of 473 patients, was retrospectively retrieved from the PACS (Picture Achieving and Communication System) of a single institution. We developed a fully connected CNN consisting of four convolutional blocks, three fully connected layers, and a single neuron as output. The model was trained and validated on 80% of the data using the mean-squared error as a cost function to minimize the difference between the predicted and reference bone age values through the Adam optimization algorithm. Data augmentation was applied to the training and validation sets yielded in doubling the data samples. The performance of the trained model was evaluated on a test data set (20%) using various metrics including, the mean absolute error (MAE), median absolute error (MedAE), root-mean-squared error (RMSE), and mean absolute percentage error (MAPE). The code of the developed model for predicting the bone age in this study is available publicly on GitHub at https://github.com/afiosman/deep-learning-based-bone-age-estimation .

RESULTS

Experimental results demonstrate the sound capabilities of our model in predicting the bone age on the left-hand radiographs as in the majority of the cases, the predicted bone ages and reference bone ages are nearly close to each other with a calculated MAE of 2.3 [1.9, 2.7; 0.95 confidence level] years, MedAE of 2.1 years, RMAE of 3.0 [1.5, 4.5; 0.95 confidence level] years, and MAPE of 0.29 (29%) on the test data set.

CONCLUSION

These findings highlight the usability of estimating the bone age from left-hand radiographs, helping radiologists to verify their own results considering the margin of error on the model. The performance of our proposed model could be improved with additional refining and validation.

Bone age assessment from articular surface and epiphysis using deep neural networks

Bone age assessment is of great significance to genetic diagnosis and endocrine diseases. Traditional bone age diagnosis mainly relies on experienced radiologists to examine the regions of interest in hand radiography, but it is time-consuming and may even lead to a vast error between the diagnosis result and the reference. The existing computer-aided methods predict bone age based on general regions of interest but do not explore specific regions of interest in hand radiography. This paper aims to solve such problems by performing bone age prediction on the articular surface and epiphysis from hand radiography using deep convolutional neural networks. The articular surface and epiphysis datasets are established from the Radiological Society of North America (RSNA) pediatric bone age challenge, where the specific feature regions of the articular surface and epiphysis are manually segmented from hand radiography. Five convolutional neural networks, i.e., ResNet50, SENet, DenseNet-121, EfficientNet-b4, and CSPNet, are employed to improve the accuracy and efficiency of bone age diagnosis in clinical applications. Experiments show that the best-performing model can yield a mean absolute error (MAE) of 7.34 months on the proposed articular surface and epiphysis datasets, which is more accurate and fast than the radiologists. The project is available at https://github.com/YameiDeng/BAANet/, and the annotated dataset is also published at https://doi.org/10.5281/zenodo.7947923.

Texture Analysis for the Bone Age Assessment from MRI Images of Adolescent Wrists in Boys

Currently, bone age is assessed by X-rays. It enables the evaluation of the child's development and is an important diagnostic factor. However, it is not sufficient to diagnose a specific disease because the diagnoses and prognoses may arise depending on how much the given case differs from the norms of bone age.

BACKGROUND

The use of magnetic resonance images (MRI) to assess the age of the patient would extend diagnostic possibilities. The bone age test could then become a routine screening test. Changing the method of determining the bone age would also prevent the patient from taking a dose of ionizing radiation, making the test less invasive.

METHODS

The regions of interest containing the wrist area and the epiphyses of the radius are marked on the magnetic resonance imaging of the non-dominant hand of boys aged 9 to 17 years. Textural features are computed for these regions, as it is assumed that the texture of the wrist image contains information about bone age.

RESULTS

The regression analysis revealed that there is a high correlation between the bone age of a patient and the MRI-derived textural features derived from MRI. For DICOM T1-weighted data, the best scores reached 0.94 R2, 0.46 RMSE, 0.21 MSE, and 0.33 MAE.

CONCLUSIONS

The experiments performed have shown that using the MRI images gives reliable results in the assessment of bone age while not exposing the patient to ionizing radiation.

A Global-Local Feature Fusion Convolutional Neural Network for Bone Age Assessment of Hand X-ray Images

Bone age assessment plays a critical role in the investigation of endocrine, genetic, and growth disorders in children. This process is usually conducted manually, with some drawbacks, such as reliance on the pediatrician’s experience and extensive labor, as well as high variations among methods. Most deep learning models use one neural network to extract the global information from the whole input image, ignoring the local details that doctors care about. In this paper, we propose a global-local feature fusion convolutional neural network, including a global pathway to capture the global contextual information and a local pathway to extract the fine-grained information from local patches. The fine-grained information is integrated into the global context information layer-by-layer to assist in predicting bone age. We evaluated the proposed method on a dataset with 11,209 X-ray images with an age range of 4–18 years. Compared with other state-of-the-art methods, the proposed global-local network reduces the mean absolute error of the estimated ages to 0.427 years for males and 0.455 years for females; the average accuracy rate is within 6 months and 12 months, reaching 70% and 91%, respectively. In addition, the effectiveness and rationality of the model were verified on a public dataset.

Improved Deep Convolutional Neural Networks via Boosting for Predicting the Quality of In Vitro Bovine Embryos

Automated diagnosis for the quality of bovine in vitro-derived embryos based on imaging data is an important research problem in developmental biology. By predicting the quality of embryos correctly, embryologists can (1) avoid the time-consuming and tedious work of subjective visual examination to assess the quality of embryos; (2) automatically perform real-time evaluation of embryos, which accelerates the examination process; and (3) possibly avoid the economic, social, and medical implications caused by poor-quality embryos. While generated embryo images provide an opportunity for analyzing such images, there is a lack of consistent noninvasive methods utilizing deep learning to assess the quality of embryos. Hence, designing high-performance deep learning algorithms is crucial for data analysts who work with embryologists. A key goal of this study is to provide advanced deep learning tools to embryologists, who would, in turn, use them as prediction calculators to evaluate the quality of embryos. The proposed deep learning approaches utilize a modified convolutional neural network, with or without boosting techniques, to improve the prediction performance. Experimental results on image data pertaining to in vitro bovine embryos show that our proposed deep learning approaches perform better than existing baseline approaches in terms of prediction performance and statistical significance.

A Vaginitis Classification Method Based on Multi-Spectral Image Feature Fusion

Vaginitis is one of the commonly encountered diseases of female reproductive tract infections. The clinical diagnosis mainly relies on manual observation under a microscope. There has been some investigation on the classification of vaginitis diseases based on computer-aided diagnosis to reduce the workload of clinical laboratory staff. However, the studies only using RGB images limit the development of vaginitis diagnosis. Through multi-spectral technology, we propose a vaginitis classification algorithm based on multi-spectral image feature layer fusion. Compared with the traditional RGB image, our approach improves the classification accuracy by 11.39%, precision by 15.82%, and recall by 27.25%. Meanwhile, we prove that the level of influence of each spectrum on the disease is distinctive, and the subdivided spectral image is more conducive to the image analysis of vaginitis disease.

How Artificial Intelligence and Machine Learning Is Assisting Us to Extract Meaning from Data on Bone Mechanics?

Dramatic advancements in interdisciplinary research with the fourth paradigm of science, especially the implementation of computer science, nourish the potential for artificial intelligence (AI), machine learning (ML), and artificial neural network (ANN) algorithms to be applied to studies concerning mechanics of bones. Despite recent enormous advancement in techniques, gaining deep knowledge to find correlations between bone shape, material, mechanical, and physical responses as well as properties is a daunting task. This is due to both complexity of the material itself and the convoluted shapes that this complex material forms. Moreover, many uncertainties and ambiguities exist concerning the use of traditional computational techniques that hinders gaining a full comprehension of this advanced biological material. This book chapter offers a review of literature on the use of AI, ML, and ANN in the study of bone mechanics research. A main question as to why to implement AI and ML in the mechanics of bones is fully addressed and explained. This chapter also introduces AI and ML and elaborates on the main features of ML algorithms such as learning paradigms, subtypes, main ideas with examples, performance metrics, training algorithms, and training datasets. As a frequently employed ML algorithm in bone mechanics, feedforward ANNs are discussed to make their taxonomy and working principles more readily comprehensible to researchers. A summary as well as detailed review of papers that employed ANNs to learn from collected data on bone mechanics are presented. Reviewing literature on the use of these data-driven tools is essential since their wider application has the potential to: improve clinical assessments enabling real-time simulations; avoid and/or minimize injuries; and, encourage early detection of such injuries in the first place.

Traditional and New Methods of Bone Age Assessment-An Overview

Bone age is one of biological indicators of maturity used in clinical practice and it is a very important parameter of a child’s assessment, especially in paediatric endocrinology. The most widely used method of bone age assessment is by performing a hand and wrist radiograph and its analysis with Greulich-Pyle or Tanner-Whitehouse atlases, although it has been about 60 years since they were published. Due to the progress in the area of Computer-Aided Diagnosis and application of artificial intelligence in medicine, lately, numerous programs for automatic bone age assessment have been created. Most of them have been verified in clinical studies in comparison to traditional methods, showing good precision while eliminating inter- and intra-rater variability and significantly reducing the time of assessment. Additionally, there are available methods for assessment of bone age which avoid X-ray exposure, using modalities such as ultrasound or magnetic resonance imaging.

Important Tools for Use by Pediatric Endocrinologists in the Assessment of Short Stature

Assessment and management of children with growth failure has improved greatly over recent years. However, there remains a strong potential for further improvements by using novel digital techniques. A panel of experts discussed developments in digitalization of a number of important tools used by pediatric endocrinologists at the third 360° European Meeting on Growth and Endocrine Disorders, funded by Merck KGaA, Germany, and this review is based on those discussions. It was reported that electronic monitoring and new algorithms have been devised that are providing more sensitive referral for short stature. In addition, computer programs have improved ways in which diagnoses are coded for use by various groups including healthcare providers and government health systems. Innovative cranial imaging techniques have been devised that are considered safer than using gadolinium contrast agents and are also more sensitive and accurate. Deep-learning neural networks are changing the way that bone age and bone health are assessed, which are more objective than standard methodologies. Models for prediction of growth response to growth hormone (GH) treatment are being improved by applying novel artificial intelligence methods that can identify non-linear and linear factors that relate to response, providing more accurate predictions. Determination and interpretation of insulin-like growth factor-1 (IGF-1) levels are becoming more standardized and consistent, for evaluation across different patient groups, and computer-learning models indicate that baseline IGF-1 standard deviation score is among the most important indicators of GH therapy response. While physicians involved in child growth and treatment of disorders resulting in growth failure need to be aware of, and keep abreast of, these latest developments, treatment decisions and management should continue to be based on clinical decisions. New digital technologies and advancements in the field should be aimed at improving clinical decisions, making greater standardization of assessment and facilitating patient-centered approaches.

Intelligent Bone Age Assessment: An Automated System to Detect a Bone Growth Problem Using Convolutional Neural Networks with Attention Mechanism

Skeletal bone age assessment using X-ray images is a standard clinical procedure to detect any anomaly in bone growth among kids and babies. The assessed bone age indicates the actual level of growth, whereby a large discrepancy between the assessed and chronological age might point to a growth disorder. Hence, skeletal bone age assessment is used to screen the possibility of growth abnormalities, genetic problems, and endocrine disorders. Usually, the manual screening is assessed through X-ray images of the non-dominant hand using the Greulich-Pyle (GP) or Tanner-Whitehouse (TW) approach. The GP uses a standard hand atlas, which will be the reference point to predict the bone age of a patient, while the TW uses a scoring mechanism to assess the bone age using several regions of interest information. However, both approaches are heavily dependent on individual domain knowledge and expertise, which is prone to high bias in inter and intra-observer results. Hence, an automated bone age assessment system, which is referred to as Attention-Xception Network (AXNet) is proposed to automatically predict the bone age accurately. The proposed AXNet consists of two parts, which are image normalization and bone age regression modules. The image normalization module will transform each X-ray image into a standardized form so that the regressor network can be trained using better input images. This module will first extract the hand region from the background, which is then rotated to an upright position using the angle calculated from the four key-points of interest. Then, the masked and rotated hand image will be aligned such that it will be positioned in the middle of the image. Both of the masked and rotated images will be obtained through existing state-of-the-art deep learning methods. The last module will then predict the bone age through the Attention-Xception network that incorporates multiple layers of spatial-attention mechanism to emphasize the important features for more accurate bone age prediction. From the experimental results, the proposed AXNet achieves the lowest mean absolute error and mean squared error of 7.699 months and 108.869 months2, respectively. Therefore, the proposed AXNet has demonstrated its potential for practical clinical use with an error of less than one year to assist the experts or radiologists in evaluating the bone age objectively.

Artificial intelligence system can achieve comparable results to experts for bone age assessment of Chinese children with abnormal growth and development

Objective

Bone age (BA) is a crucial indicator for revealing the growth and development of children. This study tested the performance of a fully automated artificial intelligence (AI) system for BA assessment of Chinese children with abnormal growth and development.

Materials and Methods

A fully automated AI system based on the Greulich and Pyle (GP) method was developed for Chinese children by using 8,000 BA radiographs from five medical centers nationwide in China. Then, a total of 745 cases (360 boys and 385 girls) with abnormal growth and development from another tertiary medical center of north China were consecutively collected between January and October 2018 to test the system. The reference standard was defined as the result interpreted by two experienced reviewers (a radiologist with 10 years and an endocrinologist with 15 years of experience in BA reading) through consensus using the GP atlas. BA accuracy within 1 year, root mean square error (RMSE), mean absolute difference (MAD), and 95% limits of agreement according to the Bland-Altman plot were statistically calculated.

Results

For Chinese pediatric patients with abnormal growth and development, the accuracy of this new automated AI system within 1 year was 84.60% as compared to the reference standard, with the highest percentage of 89.45% in the 12- to 18-year group. The RMSE, MAD, and 95% limits of agreement of the AI system were 0.76 years, 0.58 years, and -1.547 to 1.428, respectively, according to the Bland-Altman plot. The largest difference between the AI and experts' BA result was noted for patients of short stature with bone deformities, severe osteomalacia, or different rates of maturation of the carpals and phalanges.

Conclusions

The developed automated AI system could achieve comparable BA results to experienced reviewers for Chinese children with abnormal growth and development.