Applications of machine learning for imaging-driven diagnosis of musculoskeletal malignancies-a scoping review

Deep learning approaches for classification tasks in medical X-ray, MRI, and ultrasound images: a scoping review

Medical images occupy the largest part of the existing medical information and dealing with them is challenging not only in terms of management but also in terms of interpretation and analysis. Hence, analyzing, understanding, and classifying them, becomes a very expensive and time-consuming task, especially if performed manually. Deep learning is considered a good solution for image classification, segmentation, and transfer learning tasks since it offers a large number of algorithms to solve such complex problems. PRISMA-ScR guidelines have been followed to conduct the scoping review with the aim of exploring how deep learning is being used to classify a broad spectrum of diseases diagnosed using an X-ray, MRI, or Ultrasound image modality.Findings contribute to the existing research by outlining the characteristics of the adopted datasets and the preprocessing or augmentation techniques applied to them. The authors summarized all relevant studies based on the deep learning models used and the accuracy achieved for classification. Whenever possible, they included details about the hardware and software configurations, as well as the architectural components of the models employed. Moreover, the models that achieved the highest accuracy in disease classification were highlighted, along with their strengths. The authors also discussed the limitations of the current approaches and proposed future directions for medical image classification.

Semi-supervised Label Generation for 3D Multi-modal MRI Bone Tumor Segmentation

Medical image segmentation is challenging due to the need for expert annotations and the variability of these manually created labels. Previous methods tackling label variability focus on 2D segmentation and single modalities, but reliable 3D multi-modal approaches are necessary for clinical applications such as in oncology. In this paper, we propose a framework for generating reliable and unbiased labels with minimal radiologist input for supervised 3D segmentation, reducing radiologists' efforts and variability in manual labeling. Our framework generates AI-assisted labels through a two-step process involving 3D multi-modal unsupervised segmentation based on feature clustering and semi-supervised refinement. These labels are then compared against traditional expert-generated labels in a downstream task consisting of 3D multi-modal bone tumor segmentation. Two 3D-Unet models are trained, one with manually created expert labels and the other with AI-assisted labels. Following this, a blind evaluation is performed on the segmentations of these two models to assess the reliability of training labels. The framework effectively generated accurate segmentation labels with minimal expert input, achieving state-of-the-art performance. The model trained with AI-assisted labels outperformed the baseline model in 61.67% of blind evaluations, indicating the enhancement of segmentation quality and demonstrating the potential of AI-assisted labeling to reduce radiologists' workload and improve label reliability for 3D multi-modal bone tumor segmentation. The code is available at https://github.com/acurtovilalta/3D_LabelGeneration .

Pediatric Bone Tumors: Location and Age Distribution of 420 Cases

BACKGROUND/OBJECTIVES

One of the most important diagnostic tools in bone tumors is X-rays. Preliminary and, in the case of some benign lesions, definitive diagnoses are formed using this basic tool. Part of the decision making in this stage is based on statistical probability using the patient's age, as well as the incidence and predilection sites of different entities. The information used today is based on older and fragmented data. To verify the underlying principles, we retrospectively evaluated all bone tumors in children and adolescents treated by our tertiary center in the last 20 years.

METHODS

For this retrospective study, patients under the age of 18 years suffering from histopathologically verified bone tumors were evaluated. Data were retrieved from our local musculoskeletal tumor database.

RESULTS

We were able to include 420 children treated for bone tumors in our tertiary center. The cohort consisted of 335 benign and 85 malignant lesions. The most common lesions were 137 osteochondromas; the malignant tumors consisted mainly of osteosarcomas (53) and Ewing's sarcomas (28). The primary predilection sites were the metaphyses of long bones.

CONCLUSIONS

We were able to confirm and supplement the fragmentary data of these rare diseases using our own cohort.

Impact of metadata in multimodal classification of bone tumours

The accurate classification of bone tumours is crucial for guiding clinical decisions regarding treatment and follow-up. However, differentiating between various tumour types is challenging due to the rarity of certain entities, high intra-class variability, and limited training data in clinical practice. This study proposes a multimodal deep learning model that integrates clinical metadata and X-ray imaging to improve the classification of primary bone tumours. The dataset comprises 1,785 radiographs from 804 patients collected between 2000 and 2020, including metadata such as age, affected bone site, tumour position, and gender. Ten tumour types were selected, with histopathology or tumour board decisions serving as the reference standard.

METHODS

Our model is based on the NesT image classification model and a multilayer perceptron with a joint fusion architecture. Descriptive statistics included incidence and percentage ratios for discrete parameters, and mean, standard deviation, median, and interquartile range for continuous parameters.

RESULTS

The mean age of the patients was 33.62 ± 18.60 years, with 54.73% being male. Our multimodal deep learning model achieved 69.7% accuracy in classifying primary bone tumours, outperforming the Vision Transformer model by five percentage points. SHAP values indicated that age had the most substantial influence among the considered metadata.

CONCLUSION

The joint fusion approach developed in this study, integrating clinical metadata and imaging data, outperformed state-of-the-art models in classifying primary bone tumours. The use of SHAP values provided insights into the impact of different metadata on the model's performance, highlighting the significant role of age. This approach has potential implications for improving diagnostic accuracy and understanding the influence of clinical factors in tumour classification.

Recommender-based bone tumour classification with radiographs-a link to the past

OBJECTIVES

To develop an algorithm to link undiagnosed patients to previous patient histories based on radiographs, and simultaneous classification of multiple bone tumours to enable early and specific diagnosis.

MATERIALS AND METHODS

For this retrospective study, data from 2000 to 2021 were curated from our database by two orthopaedic surgeons, a radiologist and a data scientist. Patients with complete clinical and pre-therapy radiographic data were eligible. To ensure feasibility, the ten most frequent primary tumour entities, confirmed histologically or by tumour board decision, were included. We implemented a ResNet and transformer model to establish baseline results. Our method extracts image features using deep learning and then clusters the k most similar images to the target image using a hash-based nearest-neighbour recommender approach that performs simultaneous classification by majority voting. The results were evaluated with precision-at-k, accuracy, precision and recall. Discrete parameters were described by incidence and percentage ratios. For continuous parameters, based on a normality test, respective statistical measures were calculated.

RESULTS

Included were data from 809 patients (1792 radiographs; mean age 33.73 ± 18.65, range 3-89 years; 443 men), with Osteochondroma (28.31%) and Ewing sarcoma (1.11%) as the most and least common entities, respectively. The dataset was split into training (80%) and test subsets (20%). For k = 3, our model achieved the highest mean accuracy, precision and recall (92.86%, 92.86% and 34.08%), significantly outperforming state-of-the-art models (54.10%, 55.57%, 19.85% and 62.80%, 61.33%, 23.05%).

CONCLUSION

Our novel approach surpasses current models in tumour classification and links to past patient data, leveraging expert insights.

CLINICAL RELEVANCE STATEMENT

The proposed algorithm could serve as a vital support tool for clinicians and general practitioners with limited experience in bone tumour classification by identifying similar cases and classifying bone tumour entities.

KEY POINTS

• Addressed accurate bone tumour classification using radiographic features. • Model achieved 92.86%, 92.86% and 34.08% mean accuracy, precision and recall, respectively, significantly surpassing state-of-the-art models. • Enhanced diagnosis by integrating prior expert patient assessments.

Advancing musculoskeletal tumor diagnosis: Automated segmentation and predictive classification using deep learning and radiomics

OBJECTIVES

Musculoskeletal (MSK) tumors, given their high mortality rate and heterogeneity, necessitate precise examination and diagnosis to guide clinical treatment effectively. Magnetic resonance imaging (MRI) is pivotal in detecting MSK tumors, as it offers exceptional image contrast between bone and soft tissue. This study aims to enhance the speed of detection and the diagnostic accuracy of MSK tumors through automated segmentation and grading utilizing MRI.

MATERIALS AND METHODS

The research included 170 patients (mean age, 58 years ±12 (standard deviation), 84 men) with MSK lesions, who underwent MRI scans from April 2021 to May 2023. We proposed a deep learning (DL) segmentation model MSAPN based on multi-scale attention and pixel-level reconstruction, and compared it with existing algorithms. Using MSAPN-segmented lesions to extract their radiomic features for the benign and malignant classification of tumors.

RESULTS

Compared to the most advanced segmentation algorithms, MSAPN demonstrates better performance. The Dice similarity coefficients (DSC) are 0.871 and 0.815 in the testing set and independent validation set, respectively. The radiomics model for classifying benign and malignant lesions achieves an accuracy of 0.890. Moreover, there is no statistically significant difference between the radiomics model based on manual segmentation and MSAPN segmentation.

CONCLUSION

This research contributes to the advancement of MSK tumor diagnosis through automated segmentation and predictive classification. The integration of DL algorithms and radiomics shows promising results, and the visualization analysis of feature maps enhances clinical interpretability.

Deep Learning-Based Detection of Bone Tumors around the Knee in X-rays of Children

Even though tumors in children are rare, they cause the second most deaths under the age of 18 years. More often than in other age groups, underage patients suffer from malignancies of the bones, and these mostly occur in the area around the knee. One problem in the treatment is the early detection of bone tumors, especially on X-rays. The rarity and non-specific clinical symptoms further prolong the time to diagnosis. Nevertheless, an early diagnosis is crucial and can facilitate the treatment and therefore improve the prognosis of affected children. A new approach to evaluating X-ray images using artificial intelligence may facilitate the detection of suspicious lesions and, hence, accelerate the referral to a specialized center. We implemented a Vision Transformer model for image classification of healthy and pathological X-rays. To tackle the limited amount of data, we used a pretrained model and implemented extensive data augmentation. Discrete parameters were described by incidence and percentage ratio and continuous parameters by median, standard deviation and variance. For the evaluation of the model accuracy, sensitivity and specificity were computed. The two-entity classification of the healthy control group and the pathological group resulted in a cross-validated accuracy of 89.1%, a sensitivity of 82.2% and a specificity of 93.2% for test groups. Grad-CAMs were created to ensure the plausibility of the predictions. The proposed approach, using state-of-the-art deep learning methodology to detect bone tumors on knee X-rays of children has achieved very good results. With further improvement of the algorithm, enlargement of the dataset and removal of potential biases, this could become a useful additional tool, especially to support general practitioners for early, accurate and specific diagnosis of bone lesions in young patients.