1
|
Aydin OU, Taha AA, Hilbert A, Khalil AA, Galinovic I, Fiebach JB, Frey D, Madai VI. On the usage of average Hausdorff distance for segmentation performance assessment: hidden error when used for ranking. Eur Radiol Exp 2021; 5:4. [PMID: 33474675 PMCID: PMC7817746 DOI: 10.1186/s41747-020-00200-2] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2020] [Accepted: 12/03/2020] [Indexed: 02/07/2023] Open
Abstract
Average Hausdorff distance is a widely used performance measure to calculate the distance between two point sets. In medical image segmentation, it is used to compare ground truth images with segmentations allowing their ranking. We identified, however, ranking errors of average Hausdorff distance making it less suitable for applications in segmentation performance assessment. To mitigate this error, we present a modified calculation of this performance measure that we have coined “balanced average Hausdorff distance”. To simulate segmentations for ranking, we manually created non-overlapping segmentation errors common in magnetic resonance angiography cerebral vessel segmentation as our use-case. Adding the created errors consecutively and randomly to the ground truth, we created sets of simulated segmentations with increasing number of errors. Each set of simulated segmentations was ranked using both performance measures. We calculated the Kendall rank correlation coefficient between the segmentation ranking and the number of errors in each simulated segmentation. The rankings produced by balanced average Hausdorff distance had a significantly higher median correlation (1.00) than those by average Hausdorff distance (0.89). In 200 total rankings, the former misranked 52 whilst the latter misranked 179 segmentations. Balanced average Hausdorff distance is more suitable for rankings and quality assessment of segmentations than average Hausdorff distance.
Collapse
Affiliation(s)
- Orhun Utku Aydin
- CLAIM - Charité Lab for Artificial Intelligence in Medicine, Charité Universitätsmedizin Berlin, Berlin, Germany.
| | - Abdel Aziz Taha
- Research Studio Data Science, Research Studios Austria, Salzburg, Austria
| | - Adam Hilbert
- CLAIM - Charité Lab for Artificial Intelligence in Medicine, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Ahmed A Khalil
- Centre for Stroke Research Berlin, Charité Universitätsmedizin Berlin, Berlin, Germany.,Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.,Mind, Brain, Body Institute, Berlin School of Mind and Brain, Humboldt-Universität Berlin, Berlin, Germany
| | - Ivana Galinovic
- Centre for Stroke Research Berlin, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Jochen B Fiebach
- Centre for Stroke Research Berlin, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Dietmar Frey
- CLAIM - Charité Lab for Artificial Intelligence in Medicine, Charité Universitätsmedizin Berlin, Berlin, Germany
| | - Vince Istvan Madai
- CLAIM - Charité Lab for Artificial Intelligence in Medicine, Charité Universitätsmedizin Berlin, Berlin, Germany.,School of Computing and Digital Technology, Faculty of Computing, Engineering and the Built Environment, Birmingham City University, Birmingham, UK
| |
Collapse
|
2
|
Xiao Y, Wang C, Sun Y, Zhang X, Cui L, Yu J, Zheng H. Quantitative Estimation of Passive Elastic Properties of Individual Skeletal Muscle in Vivo Using Normalized Elastic Modulus-Length Curve. IEEE Trans Biomed Eng 2020; 67:3371-3379. [DOI: 10.1109/tbme.2020.2985724] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
|
3
|
Chen C, Wang Y, Yu J, Zhou Z, Shen L, Chen YQ. Automatic motion analysis system for pyloric flow in ultrasonic videos. IEEE J Biomed Health Inform 2014; 18:130-8. [PMID: 24403410 DOI: 10.1109/jbhi.2013.2272090] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Ultrasonography has been widely used to evaluate duodenogastric reflux (DGR). But to the best of our knowledge, no automatic analysis system was developed to realize the quantitative computer-aided analysis. In this paper, we propose a system to perform the automatic detection of DGR in the ultrasonic image sequences by applying the automatic motion analysis. The motion field is estimated based on image velocimetry. Then, an intelligent motion analysis is applied. For the DGR detection, the motion and structural information is combined to analyze the transploric motion of the fluid. In order to test the performance of the proposed system, we designed the experiment with the real and synthetic ultrasonic data. The proposed system achieved a good performance in the DGR detection. The automatic results were accordant with the gold standard in analyzing the fluid motion. The proposed system is supposed to be a promising tool for the study and evaluation of DGR.
Collapse
|
4
|
Chen C, Wang Y, Yu J, Zhou Z, Shen L, Chen Y. Weighted cross-correlation based variational optical flow for gastric flow analysis in ultrasonic videos. Med Phys 2013; 40:052901. [DOI: 10.1118/1.4798978] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022] Open
|