1
|
Viana da Silva M, de Carvalho Santos N, Ouellette J, Lacoste B, Comin CH. A new dataset for measuring the performance of blood vessel segmentation methods under distribution shifts. PLoS One 2025; 20:e0322048. [PMID: 40424440 PMCID: PMC12112280 DOI: 10.1371/journal.pone.0322048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/11/2024] [Accepted: 03/15/2025] [Indexed: 05/29/2025] Open
Abstract
Creating a dataset for training supervised machine learning algorithms can be a demanding task. This is especially true for blood vessel segmentation since one or more specialists are usually required for image annotation, and creating ground truth labels for just a single image can take up to several hours. In addition, it is paramount that the annotated samples represent well the different conditions that might affect the imaged tissues as well as possible changes in the image acquisition process. This can only be achieved by considering samples that are typical in the dataset as well as atypical, or even outlier, samples. We introduce VessMAP, an annotated and highly heterogeneous blood vessel segmentation dataset acquired by carefully sampling relevant images from a large non-annotated dataset containing fluorescence microscopy images. Each image of the dataset contains metadata information regarding the contrast, amount of noise, density, and intensity variability of the vessels. Prototypical and atypical samples were carefully selected from the base dataset using the available metadata information, thus defining an assorted set of images that can be used for measuring the performance of segmentation algorithms on samples that are highly distinct from each other. We show that datasets traditionally used for developing new blood vessel segmentation algorithms tend to have low heterogeneity. Thus, neural networks trained on as few as four samples can generalize well to all other samples. In contrast, the training samples used for the VessMAP dataset can be critical to the generalization capability of a neural network. For instance, training on samples with good contrast leads to models with poor inference quality. Interestingly, while some training sets lead to Dice scores as low as 0.59, a careful selection of the training samples results in a Dice score of 0.85. Thus, the VessMAP dataset can be used for the development of new active learning methods for selecting relevant samples for manual annotation as well as for analyzing the robustness of segmentation models to distribution shifts of the data.
Collapse
Affiliation(s)
| | | | - Julie Ouellette
- Department of Cellular and Molecular Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada
- Neuroscience Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Baptiste Lacoste
- Department of Cellular and Molecular Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada
- Neuroscience Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Cesar H. Comin
- Department of Computer Science, Federal University of S ao Carlos, São Carlos, Brazil
| |
Collapse
|
2
|
Liu S, Shang B, Yan J, Zhu Z, Ding Y, Zhou Q, Wei C, Shen Y, Zhu X. A method for quantifying and automatic grading of musculoskeletal ultrasound superb microvascular imaging based on dynamic analysis of optical flow model. Sci Rep 2025; 15:13369. [PMID: 40247053 PMCID: PMC12006311 DOI: 10.1038/s41598-025-97924-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2024] [Accepted: 04/08/2025] [Indexed: 04/19/2025] Open
Abstract
Superb microvascular flow signals in joints are important indicators for evaluating inflammation in arthritis diagnosis. Super Microvascular Imaging (SMI), a musculoskeletal ultrasound technique, captures microvascular signals with enhanced resolution, enabling improved quantitative analysis of joint superb microvascular flow. However, existing musculoskeletal ultrasound imaging predominantly relies on static observations for analyzing these signals, which are heavily influenced by subjective factors, thereby limiting diagnostic accuracy for arthritis. This study introduces a novel quantitative and automated grading method utilizing dynamic analysis through an optical flow model. Real-time dynamic quantification of superb microvascular flow signals is achieved via motion estimation and skeleton extraction based on the optical flow model. The Kappa consistency test evaluates the agreement between the automated grading system and physician assessments, with differences between the two methods analyzed. A total of 47 patient samples were included, comprising 20 males and 27 females (p = 0.307 > 0.05, χ2=1.042). The agreement between the automated grading system and physician assessments reached 70.2%, with a Kappa value of 0.627 (p < 0.001), indicating good consistency. Nonetheless, the system displayed a tendency to high-grade cases of moderate inflammation. The proposed quantitative and automated grading method for superb microvascular flow, based on dynamic analysis through an optical flow model, improves the objectivity and consistency of superb microvascular flow grading and demonstrates significant clinical potential. The method shows strong anti-interference performance in noisy signal environments, representing a promising advancement for non-invasive arthritis diagnosis.
Collapse
Affiliation(s)
- Shanna Liu
- Department of Information Technology, the Fourth Affiliated Hospital of School of Medicine, and International School of Medicine, International Institutes of Medicine, Zhejiang University, Yiwu, 322000, China
| | - Bo Shang
- College of Medical Engineering and Technology, Xinjiang Medical University, Urumqi, 830017, Xinjiang, China
| | - Junliang Yan
- Department of Ultrasound in Medicine, the Fourth Affiliated Hospital of School of Medicine, and International School of Medicine, International Institutes of Medicine, Zhejiang University, Yiwu, 322000, China
| | - Zenghua Zhu
- School of Information Engineering, Hangzhou Medical College, Hangzhou, Zhejiang, 311399, China
| | - Yuanhao Ding
- Department of Information Technology, the Fourth Affiliated Hospital of School of Medicine, and International School of Medicine, International Institutes of Medicine, Zhejiang University, Yiwu, 322000, China
| | - Qingli Zhou
- Department of Information Technology, the Fourth Affiliated Hospital of School of Medicine, and International School of Medicine, International Institutes of Medicine, Zhejiang University, Yiwu, 322000, China
| | - Chengjing Wei
- College of Public Health, Xinjiang Medical University, Urumqi, 830017, Xinjiang, China
| | - Yuqiang Shen
- Department of Information Technology, the Fourth Affiliated Hospital of School of Medicine, and International School of Medicine, International Institutes of Medicine, Zhejiang University, Yiwu, 322000, China.
| | - Xinjian Zhu
- Department of Information Technology, the Fourth Affiliated Hospital of School of Medicine, and International School of Medicine, International Institutes of Medicine, Zhejiang University, Yiwu, 322000, China.
| |
Collapse
|