Self-Supervised Pretext Tasks in Model Robustness & Generalizability: A Revisit from Medical Imaging Perspective

Positional contrastive learning for improved thigh muscle segmentation in MR images

The accurate segmentation of individual muscles is essential for quantitative MRI analysis of thigh images. Deep learning methods have achieved state-of-the-art results in segmentation, but they require large numbers of labeled data to perform well. However, labeling individual thigh muscles slice by slice for numerous volumes is a laborious and time-consuming task, which limits the availability of annotated datasets. To address this challenge, self-supervised learning (SSL) emerges as a promising technique to enhance model performance by pretraining the model on unlabeled data. A recent approach, called positional contrastive learning, exploits the information given by the axial position of the slices to learn features transferable on the segmentation task. The aim of this work was to propose positional contrastive SSL for the segmentation of individual thigh muscles from MRI acquisitions in a population of elderly healthy subjects and to evaluate it on different levels of limited annotated data. An unlabeled dataset of 72 T1w MRI thigh acquisitions was available for SSL pretraining, while a labeled dataset of 52 volumes was employed for the final segmentation task, split into training and test sets. The effectiveness of SSL pretraining to fine-tune a U-Net architecture for thigh muscle segmentation was compared with that of a randomly initialized model (RND), considering an increasing number of annotated volumes (S = 1, 2, 5, 10, 20, 30, 40). Our results demonstrated that SSL yields substantial improvements in Dice similarity coefficient (DSC) when using a very limited number of labeled volumes (e.g., forS $$ S $$  = 1, DSC 0.631 versus 0.530 for SSL and RND, respectively). Moreover, enhancements are achievable even when utilizing the full number of labeled subjects, with DSC = 0.927 for SSL and 0.924 for RND. In conclusion, positional contrastive SSL was effective in obtaining more accurate thigh muscle segmentation, even with a very low number of labeled data, with a potential impact of speeding up the annotation process in clinics.

How Well Do Self-Supervised Models Transfer to Medical Imaging?

Self-supervised learning approaches have seen success transferring between similar medical imaging datasets, however there has been no large scale attempt to compare the transferability of self-supervised models against each other on medical images. In this study, we compare the generalisability of seven self-supervised models, two of which were trained in-domain, against supervised baselines across eight different medical datasets. We find that ImageNet pretrained self-supervised models are more generalisable than their supervised counterparts, scoring up to 10% better on medical classification tasks. The two in-domain pretrained models outperformed other models by over 20% on in-domain tasks, however they suffered significant loss of accuracy on all other tasks. Our investigation of the feature representations suggests that this trend may be due to the models learning to focus too heavily on specific areas.