Accuracy and self-validation of automated bone age determination

Applicability and robustness of an artificial intelligence-based assessment for Greulich and Pyle bone age in a German cohort

PURPOSE

 The determination of bone age (BA) based on the hand and wrist, using the 70-year-old Greulich and Pyle (G&P) atlas, remains a widely employed practice in various institutions today. However, a more recent approach utilizing artificial intelligence (AI) enables automated BA estimation based on the G&P atlas. Nevertheless, AI-based methods encounter limitations when dealing with images that deviate from the standard hand and wrist projections. Generally, the extent to which BA, as determined by the G&P atlas, corresponds to the chronological age (CA) of a contemporary German population remains a subject of continued discourse. This study aims to address two main objectives. Firstly, it seeks to investigate whether the G&P atlas, as applied by the AI software, is still relevant for healthy children in Germany today. Secondly, the study aims to assess the performance of the AI software in handling non-strict posterior-anterior (p. a.) projections of the hand and wrist.

MATERIALS AND METHODS

 The AI software retrospectively estimated the BA in children who had undergone radiographs of a single hand using posterior-anterior and oblique planes. The primary purpose was to rule out any osseous injuries. The prediction error of BA in relation to CA was calculated for each plane and between the two planes.

RESULTS

 A total of 1253 patients (aged 3 to 16 years, median age 10.8 years, 55.7 % male) were included in the study. The average error of BA in posterior-anterior projections compared to CA was 3.0 (± 13.7) months for boys and 1.7 (± 13.7) months for girls. Interestingly, the deviation from CA tended to be even slightly lower in oblique projections than in posterior-anterior projections. The mean error in the posterior-anterior projection plane was 2.5 (± 13.7) months, while in the oblique plane it was 1.8 (± 13.9) months (p = 0.01).

CONCLUSION

 The AI software for BA generally corresponds to the age of the contemporary German population under study, although there is a noticeable prediction error, particularly in younger children. Notably, the software demonstrates robust performance in oblique projections.

KEY POINTS

  · Bone age, as determined by artificial intelligence, aligns with the chronological age of the contemporary German cohort under study.. · As determined by artificial intelligence, bone age is remarkably robust, even when utilizing oblique X-ray projections..

CITATION FORMAT

· Pape J, Hirsch F, Deffaa O et al. Applicability and robustness of an artificial intelligence-based assessment for Greulich and Pyle bone age in a German cohort. Fortschr Röntgenstr 2024; 196: 600 - 606.

Deep learning-based identification of spine growth potential on EOS radiographs

OBJECTIVES

To develop an automatic computer-based method that can help clinicians in assessing spine growth potential based on EOS radiographs.

METHODS

We developed a deep learning-based (DL) algorithm that can mimic the human judgment process to automatically determine spine growth potential and the Risser sign based on full-length spine EOS radiographs. A total of 3383 EOS cases were collected and used for the training and test of the algorithm. Subsequently, the completed DL algorithm underwent clinical validation on an additional 440 cases and was compared to the evaluations of four clinicians.

RESULTS

Regarding the Risser sign, the weighted kappa value of our DL algorithm was 0.933, while that of the four clinicians ranged from 0.909 to 0.930. In the assessment of spine growth potential, the kappa value of our DL algorithm was 0.944, while the kappa values of the four clinicians were 0.916, 0.934, 0.911, and 0.920, respectively. Furthermore, our DL algorithm obtained a slightly higher accuracy (0.973) and Youden index (0.952) compared to the best values achieved by the four clinicians. In addition, the speed of our DL algorithm was 15.2 ± 0.3 s/40 cases, much faster than the inference speeds of the clinicians, ranging from 177.2 ± 28.0 s/40 cases to 241.2 ± 64.1 s/40 cases.

CONCLUSIONS

Our algorithm demonstrated comparable or even better performance compared to clinicians in assessing spine growth potential. This stable, efficient, and convenient algorithm seems to be a promising approach to assist doctors in clinical practice and deserves further study.

CLINICAL RELEVANCE STATEMENT

This method has the ability to quickly ascertain the spine growth potential based on EOS radiographs, and it holds promise to provide assistance to busy doctors in certain clinical scenarios.

KEY POINTS

• In the clinic, there is no available computer-based method that can automatically assess spine growth potential. • We developed a deep learning-based method that could automatically ascertain spine growth potential. • Compared with the results of the clinicians, our algorithm got comparable results.

Artificial intelligence in paediatric endocrinology: conflict or cooperation

Artificial intelligence (AI) in medicine is transforming healthcare by automating system tasks, assisting in diagnostics, predicting patient outcomes and personalising patient care, founded on the ability to analyse vast datasets. In paediatric endocrinology, AI has been developed for diabetes, for insulin dose adjustment, detection of hypoglycaemia and retinopathy screening; bone age assessment and thyroid nodule screening; the identification of growth disorders; the diagnosis of precocious puberty; and the use of facial recognition algorithms in conditions such as Cushing syndrome, acromegaly, congenital adrenal hyperplasia and Turner syndrome. AI can also predict those most at risk from childhood obesity by stratifying future interventions to modify lifestyle. AI will facilitate personalised healthcare by integrating data from 'omics' analysis, lifestyle tracking, medical history, laboratory and imaging, therapy response and treatment adherence from multiple sources. As data acquisition and processing becomes fundamental, data privacy and protecting children's health data is crucial. Minimising algorithmic bias generated by AI analysis for rare conditions seen in paediatric endocrinology is an important determinant of AI validity in clinical practice. AI cannot create the patient-doctor relationship or assess the wider holistic determinants of care. Children have individual needs and vulnerabilities and are considered in the context of family relationships and dynamics. Importantly, whilst AI provides value through augmenting efficiency and accuracy, it must not be used to replace clinical skills.

Comparison of Commercial AI Software Performance for Radiograph Lung Nodule Detection and Bone Age Prediction

Background Multiple commercial artificial intelligence (AI) products exist for assessing radiographs; however, comparable performance data for these algorithms are limited. Purpose To perform an independent, stand-alone validation of commercially available AI products for bone age prediction based on hand radiographs and lung nodule detection on chest radiographs. Materials and Methods This retrospective study was carried out as part of Project AIR. Nine of 17 eligible AI products were validated on data from seven Dutch hospitals. For bone age prediction, the root mean square error (RMSE) and Pearson correlation coefficient were computed. The reference standard was set by three to five expert readers. For lung nodule detection, the area under the receiver operating characteristic curve (AUC) was computed. The reference standard was set by a chest radiologist based on CT. Randomized subsets of hand (n = 95) and chest (n = 140) radiographs were read by 14 and 17 human readers, respectively, with varying experience. Results Two bone age prediction algorithms were tested on hand radiographs (from January 2017 to January 2022) in 326 patients (mean age, 10 years ± 4 [SD]; 173 female patients) and correlated strongly with the reference standard (r = 0.99; P < .001 for both). No difference in RMSE was observed between algorithms (0.63 years [95% CI: 0.58, 0.69] and 0.57 years [95% CI: 0.52, 0.61]) and readers (0.68 years [95% CI: 0.64, 0.73]). Seven lung nodule detection algorithms were validated on chest radiographs (from January 2012 to May 2022) in 386 patients (mean age, 64 years ± 11; 223 male patients). Compared with readers (mean AUC, 0.81 [95% CI: 0.77, 0.85]), four algorithms performed better (AUC range, 0.86-0.93; P value range, <.001 to .04). Conclusions Compared with human readers, four AI algorithms for detecting lung nodules on chest radiographs showed improved performance, whereas the remaining algorithms tested showed no evidence of a difference in performance. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Omoumi and Richiardi in this issue.

Evaluation of height prediction models: from traditional methods to artificial intelligence

BACKGROUND

Traditional methods for predicting adult height (AHP) rely on manual readings of bone age (BA). However, the incorporation of artificial intelligence has recently improved the accuracy of BA readings and their incorporation into AHP models.

METHODS

This study aimed to identify the AHP model that fits the current average height for adults in Mexico. Using a cross-sectional design, the study included 1173 participants (5-18 yr). BA readings were done by two experts (manually) and with an automated method (BoneXpert®). AHP was carried out using both traditional and automated methods. The best AHP model was the one that was closest to the population mean.

RESULTS

All models overestimated the population mean (males: 0.7-6.7 cm, females: 0.9-3.7 cm). The AHP models with the smallest difference were BoneXpert for males and Bayley & Pinneau for females. However, the manual readings of BA showed significant interobserver variability (up to 43% of predictions between observers exceeded 5 cm using the Bayley & Pinneau method).

CONCLUSION

Traditional AHP models relying on manual BA readings have high interobserver variability. Therefore, BoneXpert is the most reliable option, reducing such variability and providing AHP models that remain close to the mean population height.

IMPACT

Traditional models for predicting adult height often result in overestimated height predictions. The manual reading of bone age is prone to interobserver variability, which can introduce significant biases in the prediction of adult height. The BoneXpert method minimizes the variability associated with traditional methods and demonstrates consistent results in relation to the average height of the population. This study is the first to assess adult height prediction models specifically in the current generations of Mexican children.

How AI May Transform Musculoskeletal Imaging

While musculoskeletal imaging volumes are increasing, there is a relative shortage of subspecialized musculoskeletal radiologists to interpret the studies. Will artificial intelligence (AI) be the solution? For AI to be the solution, the wide implementation of AI-supported data acquisition methods in clinical practice requires establishing trusted and reliable results. This implementation will demand close collaboration between core AI researchers and clinical radiologists. Upon successful clinical implementation, a wide variety of AI-based tools can improve the musculoskeletal radiologist's workflow by triaging imaging examinations, helping with image interpretation, and decreasing the reporting time. Additional AI applications may also be helpful for business, education, and research purposes if successfully integrated into the daily practice of musculoskeletal radiology. The question is not whether AI will replace radiologists, but rather how musculoskeletal radiologists can take advantage of AI to enhance their expert capabilities.

Deeplasia: deep learning for bone age assessment validated on skeletal dysplasias

BACKGROUND

Skeletal dysplasias collectively affect a large number of patients worldwide. Most of these disorders cause growth anomalies. Hence, evaluating skeletal maturity via the determination of bone age (BA) is a useful tool. Moreover, consecutive BA measurements are crucial for monitoring the growth of patients with such disorders, especially for timing hormonal treatment or orthopedic interventions. However, manual BA assessment is time-consuming and suffers from high intra- and inter-rater variability. This is further exacerbated by genetic disorders causing severe skeletal malformations. While numerous approaches to automate BA assessment have been proposed, few are validated for BA assessment on children with skeletal dysplasias.

OBJECTIVE

We present Deeplasia, an open-source prior-free deep-learning approach designed for BA assessment specifically validated on patients with skeletal dysplasias.

MATERIALS AND METHODS

We trained multiple convolutional neural network models under various conditions and selected three to build a precise model ensemble. We utilized the public BA dataset from the Radiological Society of North America (RSNA) consisting of training, validation, and test subsets containing 12,611, 1,425, and 200 hand and wrist radiographs, respectively. For testing the performance of our model ensemble on dysplastic hands, we retrospectively collected 568 radiographs from 189 patients with molecularly confirmed diagnoses of seven different genetic bone disorders including achondroplasia and hypochondroplasia. A subset of the dysplastic cohort (149 images) was used to estimate the test-retest precision of our model ensemble on longitudinal data.

RESULTS

The mean absolute difference of Deeplasia for the RSNA test set (based on the average of six different reference ratings) and dysplastic set (based on the average of two different reference ratings) were 3.87 and 5.84 months, respectively. The test-retest precision of Deeplasia on longitudinal data (2.74 months) is estimated to be similar to a human expert.

CONCLUSION

We demonstrated that Deeplasia is competent in assessing the age and monitoring the development of both normal and dysplastic bones.

A normative framework for artificial intelligence as a sociotechnical system in healthcare

Artificial intelligence (AI) tools are of great interest to healthcare organizations for their potential to improve patient care, yet their translation into clinical settings remains inconsistent. One of the reasons for this gap is that good technical performance does not inevitably result in patient benefit. We advocate for a conceptual shift wherein AI tools are seen as components of an intervention ensemble. The intervention ensemble describes the constellation of practices that, together, bring about benefit to patients or health systems. Shifting from a narrow focus on the tool itself toward the intervention ensemble prioritizes a "sociotechnical" vision for translation of AI that values all components of use that support beneficial patient outcomes. The intervention ensemble approach can be used for regulation, institutional oversight, and for AI adopters to responsibly and ethically appraise, evaluate, and use AI tools.

Artificial Intelligence (AI) in pediatric endocrinology

Artificial Intelligence (AI) is integrating itself throughout the medical community. AI's ability to analyze complex patterns and interpret large amounts of data will have considerable impact on all areas of medicine, including pediatric endocrinology. In this paper, we review and update the current studies of AI in pediatric endocrinology. Specific topics that are addressed include: diabetes management, bone growth, metabolism, obesity, and puberty. Becoming knowledgeable and comfortable with AI will assist pediatric endocrinologists, the goal of the paper.

Precision and Accuracy of Radiological Bone Age Assessment in Children among Different Ethnic Groups: A Systematic Review

AIM

The aim was to identify, evaluate, and summarize the findings of relevant individual studies on the precision and accuracy of radiological BA assessment procedures among children from different ethnic groups.

MATERIALS AND METHODS

A qualitative systematic review was carried out following the MOOSE statement and previously registered in PROSPERO (CRD42023449512). A search was performed in MEDLINE (PubMed) (n = 561), the Cochrane Library (n = 261), CINAHL (n = 103), Web of Science (WOS) (n = 181), and institutional repositories (n = 37) using MeSH and free terms combined with the Booleans "AND" and "OR". NOS and ROBINS-E were used to assess the methodological quality and the risk of bias of the included studies, respectively.

RESULTS

A total of 51 articles (n = 20,100) on radiological BA assessment procedures were precise in terms of intra-observer and inter-observer reliability for all ethnic groups. In Caucasian and Hispanic children, the Greulich-Pyle Atlas (GPA) was accurate at all ages, but in youths, Tanner-Whitehouse radius-ulna-short bones 3 (TW3-RUS) could be an alternative. In Asian and Arab subjects, GPA and Tanner-Whitehouse 3 (TW3) overestimated the BA in adolescents near adulthood. In African youths, GPA overestimated the BA while TW3 was more accurate.

CONCLUSION

GPA and TW3 radiological BA assessment procedures are both precise but their accuracy in estimating CA among children of different ethnic groups can be altered by racial bias.

Validation of a deep learning algorithm for bone age estimation among patients in the city of São Paulo, Brazil

Objective

To validate a deep learning (DL) model for bone age estimation in individuals in the city of São Paulo, comparing it with the Greulich and Pyle method.

Materials and Methods

This was a cross-sectional study of hand and wrist radiographs obtained for the determination of bone age. The manual analysis was performed by an experienced radiologist. The model used was based on a convolutional neural network that placed third in the 2017 Radiological Society of North America challenge. The mean absolute error (MAE) and the root-mean-square error (RMSE) were calculated for the model versus the radiologist, with comparisons by sex, race, and age.

Results

The sample comprised 714 examinations. There was a correlation between the two methods, with a coefficient of determination of 0.94. The MAE of the predictions was 7.68 months, and the RMSE was 10.27 months. There were no statistically significant differences between sexes or among races (p > 0.05). The algorithm overestimated bone age in younger individuals (p = 0.001).

Conclusion

Our DL algorithm demonstrated potential for estimating bone age in individuals in the city of São Paulo, regardless of sex and race. However, improvements are needed, particularly in relation to its use in younger patients.

The promise and limitations of artificial intelligence in musculoskeletal imaging

With the recent developments in deep learning and the rapid growth of convolutional neural networks, artificial intelligence has shown promise as a tool that can transform several aspects of the musculoskeletal imaging cycle. Its applications can involve both interpretive and non-interpretive tasks such as the ordering of imaging, scheduling, protocoling, image acquisition, report generation and communication of findings. However, artificial intelligence tools still face a number of challenges that can hinder effective implementation into clinical practice. The purpose of this review is to explore both the successes and limitations of artificial intelligence applications throughout the muscuskeletal imaging cycle and to highlight how these applications can help enhance the service radiologists deliver to their patients, resulting in increased efficiency as well as improved patient and provider satisfaction.

A comprehensive validation study of the latest version of BoneXpert on a large cohort of Caucasian children and adolescents

INTRODUCTION

Automated bone age assessment has recently become increasingly popular. The aim of this study was to assess the agreement between automated and manual evaluation of bone age using the method according to Tanner-Whitehouse (TW3) and Greulich-Pyle (GP).

METHODS

We evaluated 1285 bone age scans from 1202 children (657 scans from 612 boys) by using both manual and automated (TW3 as well as GP) bone age assessment. BoneXpert software versions 2.4.5.1. (BX2) and 3.2.1. (BX3) (Visiana, Holte, Denmark) were compared with manual evaluation using root mean squared error (RMSE) analysis.

RESULTS

RMSE for BX2 was 0.57 and 0.55 years in boys and 0.72 and 0.59 years in girls, respectively for TW3 and GP. For BX3, RMSE was 0.51 and 0.68 years in boys and 0.49 and 0.52 years in girls, respectively for TW3 and GP. Sex- and age-specific analysis for BX2 identified the largest differences between manual and automated TW3 evaluation in girls between 6-7, 12-13, 13-14 and 14-15 years, with RMSE 0.88, 0.81, 0.92 and 0.84 years, respectively. The BX3 version showed better agreement with manual TW3 evaluation (RMSE 0.64, 0.45, 0.46 and 0.57).

CONCLUSION

The latest version of the BoneXpert software provides improved and clinically sufficient agreement with manual bone age evaluation in children of both sexes compared to the previous version and may be used for routine bone age evaluation in non-selected cases in pediatric endocrinology care.

Ultrasound Imaging-Based Methods for Assessing Biological Maturity during Adolescence and Possible Application in Youth Sport: A Scoping Review

Bone maturity is an indicator for estimating the biological maturity of an individual. During adolescence, individuals show heterogeneous growth rates, and thus, differences in biological maturity should be considered in talent identification and development. Radiography of the left hand and wrist is considered the gold standard of biological maturity estimation. The use of ultrasound imaging (US) may be advantageous; however, its validity and reliability are under discussion. The aims of this scoping review are (1) to summarize the different methods for estimating biological maturity by US imaging in adolescents, (2) to obtain an overview of the level of validity and reliability of the methods, and (3) to point out the practicability and usefulness of ultrasound imaging in the field of youth sports. The search included articles published up to November 2022. The inclusion criteria stipulated that participants had to fall within the age range of 8 to 23 years and be free of bone disease and fractures in the region of interest. Nine body regions were investigated, while the hand and wrist were most commonly analyzed. US assessment methods were usually based on the estimation of a bone maturity stage, rather than a decimal bone age. Furthermore, 70% of the assessments were evaluated as applicable, 10% expressed restraint about implementation, and 20% were evaluated as not applicable. When tested, inter- and intra-rater reliability was high to excellent. Despite the absence of ionization, low costs, fast assessment, and accessibility, none of the US assessments could be referred to as a gold standard. If further development succeeds, its application has the potential to incorporate biological age into selection processes. This would allow for more equal opportunities in talent selection and thus make talent development fairer and more efficient.

A comparison of bone age assessments using automated and manual methods in children of Indian ethnicity

BACKGROUND

Bone age is useful for pediatric endocrinologists in evaluating various disorders related to growth and puberty. Traditional methods of bone age assessment, namely Greulich and Pyle (GP) and Tanner-Whitehouse (TW), have intra- and interobserver variations. Use of computer-automated methods like BoneXpert might overcome these subjective variations.

OBJECTIVE

The aim of our study was to assess the validity of BoneXpert in comparison to manual GP and TW methods for assessing bone age in children of Asian Indian ethnicity.

MATERIALS AND METHODS

We extracted from a previous study the deidentified left hand radiographs of 920 healthy children aged 2-19 years. We compared bone age as determined by four well-trained manual raters using GP and TW methods with the BoneXpert ratings. We computed accuracy using root mean square error (RMSE) to assess how close the bone age estimated by BoneXpert was to the reference rating.

RESULTS

The standard deviations (SDs) of rating among the four manual raters were 0.52 years, 0.52 years and 0.47 years for GP, TW2 and TW3 methods, respectively. The RMSEs between the automated bone age estimates and the true ratings were 0.39 years, 0.41 years and 0.36 years, respectively, for the same methods. The RMSE values were significantly lower in girls than in boys (0.53, 0.5 and 0.47 vs. 0.39, 0.47 and 0.4) by all the methods; however, no such difference was noted in classification by body mass index. The best agreement between BoneXpert and manual rating was obtained by using 50% weight on carpals (GP50). The carpal bone age was retarded in Indian children, more so in boys.

CONCLUSION

BoneXpert was accurate and performed well in estimating bone age by both GP and TW methods in healthy Asian Indian children; the error was larger in boys. The GP50 establishes "backward compatibility" with manual rating.