Exploring Brushlet Based 3D Textures in Transfer Function Specification for Direct Volume Rendering of Abdominal Organs

DECODE-3DViz: Efficient WebGL-Based High-Fidelity Visualization of Large-Scale Images using Level of Detail and Data Chunk Streaming

The DECODE-3DViz pipeline represents a major advancement in the web-based visualization of large-scale medical imaging data, particularly for peripheral artery computed tomography images. This research addresses the critical challenges of rendering high-resolution volumetric datasets via WebGL technology. By integrating progressive chunk streaming and level of detail (LOD) algorithms, DECODE-3DViz optimizes the rendering process for real-time interaction and high-fidelity visualization. The system efficiently manages WebGL texture size constraints and browser memory limitations, ensuring smooth performance even with extensive datasets. A comparative evaluation against state-of-the-art visualization tools demonstrates DECODE-3DViz's superior performance, achieving up to a 98% reduction in rendering time compared with that of competitors and maintaining a high frame rate of up to 144 FPS. Furthermore, the system exhibits exceptional GPU memory efficiency, utilizing as little as 2.6 MB on desktops, which is significantly less than the over 100 MB required by other tools. User feedback, collected through a comprehensive questionnaire, revealed high satisfaction with the tool's performance, particularly in areas such as structure definition and diagnostic capability, with an average score of 4.3 out of 5. These enhancements enable detailed and accurate visualizations of the peripheral vasculature, improving diagnostic accuracy and supporting better clinical outcomes. The DECODE-3DViz tool is open source and can be accessed at https://github.com/mohammed-abo-arab/3D_WebGL_VolumeRendering.git .

TSCA-Net: Transformer based spatial-channel attention segmentation network for medical images

Deep learning architectures based on convolutional neural network (CNN) and Transformer have achieved great success in medical image segmentation. Models based on the encoder-decoder framework like U-Net have been successfully employed in many realistic scenarios. However, due to the low contrast between object and background, various shapes and scales of objects, and complex background in medical images, it is difficult to locate targets and obtain better segmentation performance by extracting effective information from images. In this paper, an encoder-decoder architecture based on spatial and channel attention modules built by Transformer is proposed for medical image segmentation. Concretely, spatial and channel attention modules based on Transformer are utilized to extract spatial and channel global complementary information at different layers in U-shape network, which is beneficial to learn the detail features in different scales. To fuse better spatial and channel information from Transformer features, a spatial and channel feature fusion block is designed for the decoder. The proposed network inherits the advantages of both CNN and Transformer with the local feature representation and long-range dependency for medical images. Qualitative and quantitative experiments demonstrate that the proposed method outperforms against eight state-of-the-art segmentation methods on five publicly medical image datasets including different modalities, such as 80.23% and 93.56% Dice value, 67.13% and 88.94% Intersection over Union (IoU) value on the Multi-organ Nucleus Segmentation (MoNuSeg) and Combined Healthy Abdominal Organ Segmentation with Computed Tomography scans (CHAOS-CT) datasets.

Explainable multi-module semantic guided attention based network for medical image segmentation

Automated segmentation of medical images is crucial for disease diagnosis and treatment planning. Medical image segmentation has been improved based on the convolutional neural networks (CNNs) models. Unfortunately, they are still limited by scenarios in which the segmentation objective has large variations in size, boundary, position, and shape. Moreover, current CNNs have low explainability, restricting their use in clinical decisions. In this paper, we involve substantial use of various attentions in a CNN model and present an explainable multi-module semantic guided attention based network (MSGA-Net) for explainable and highly accurate medical image segmentation, which involves considering the most significant spatial regions, boundaries, scales, and channels. Specifically, we present a multi-scale attention module (MSA) to extract the most salient features at various scales from medical images. Then, we propose a semantic region-guided attention mechanism (SRGA) including location attention (LAM), channel-wise attention (CWA), and edge attention (EA) modules to extract the most important spatial, channel-wise, boundary-related features for interested regions. Moreover, we present a sequence of fine-tuning steps with the SRGA module to gradually weight the significance of interesting regions while simultaneously reducing the noise. In this work, we experimented with three different types of medical images such as dermoscopic images (HAM10000 dataset), multi-organ CT images (CHAOS 2019 dataset), and Brain tumor MRI images (BraTS 2020 dataset). Extensive experiments on all types of medical images revealed that our proposed MSGA-Net substantially increased the overall performance of all metrics over the existing models. Moreover, displaying the attention feature maps has more explainability than state-of-the-art models.

Multi-Scale Self-Guided Attention for Medical Image Segmentation

Even though convolutional neural networks (CNNs) are driving progress in medical image segmentation, standard models still have some drawbacks. First, the use of multi-scale approaches, i.e., encoder-decoder architectures, leads to a redundant use of information, where similar low-level features are extracted multiple times at multiple scales. Second, long-range feature dependencies are not efficiently modeled, resulting in non-optimal discriminative feature representations associated with each semantic class. In this paper we attempt to overcome these limitations with the proposed architecture, by capturing richer contextual dependencies based on the use of guided self-attention mechanisms. This approach is able to integrate local features with their corresponding global dependencies, as well as highlight interdependent channel maps in an adaptive manner. Further, the additional loss between different modules guides the attention mechanisms to neglect irrelevant information and focus on more discriminant regions of the image by emphasizing relevant feature associations. We evaluate the proposed model in the context of semantic segmentation on three different datasets: abdominal organs, cardiovascular structures and brain tumors. A series of ablation experiments support the importance of these attention modules in the proposed architecture. In addition, compared to other state-of-the-art segmentation networks our model yields better segmentation performance, increasing the accuracy of the predictions while reducing the standard deviation. This demonstrates the efficiency of our approach to generate precise and reliable automatic segmentations of medical images. Our code is made publicly available at: https://github.com/sinAshish/Multi-Scale-Attention.

Reversible 3D compression of segmented medical volumes: usability analysis for teleradiology and storage

BACKGROUND

DICOM standard does not have modules that provide the possibilities of two-dimensional Presentation States to three-dimensional (3D). Once the final 3D rendering is obtained, only video/image exporting or snapshots can be used. To increase the utility of 3D Presentation States in clinical practice and teleradiology, the storing and transferring the segmentation results, obtained after tedious procedures, can be very effective.

PURPOSE

To propose a strategy for preserving interaction and mobility of visualizations for teleradiology by storing and transferring only binary segmented data, which is effectively compressed by modern adaptive and context-based reversible methods.

MATERIAL AND METHODS

A diverse set of segmented data, which include four abdominal organs (liver, spleen, right, and left kidneys) from 20 T1-DUAL and 20 T2-SPIR MRI, liver from 20 CT, and abdominal aorta with aneurysms (AAA) from 19 computed tomography-angiography datasets, are collected. Each organ is segmented manually by expert physicians, and binary volumes are created. The well-established reversible binary compression methods PNG, JPEG-LS, JPEG-XR, CCITT-G4, LZW, JBIG2, and ZIP are applied to medical datasets. Recently proposed context-based (3D-RLE) and adaptive (ABIC) algorithms are also employed. The performance assessment has been presented in terms of the compression ratio that is a universal compression metric.

RESULTS

Reversible compression of binary volumes results with substantial decreases in file size such as 254 to 2.14 MB for CT-AAA, 56.7 to 0.3 MB for CT-liver. Moreover, compared to the performance of well-established methods (i.e., mean 76.14%), CR is observed to be increased significantly for all segmented organs from both CT and MRI datasets when ABIC (95.49%) and 3D-RLE (94.98%) are utilized. The hypothesis is that morphological coherence of scanning procedure and adaptation between the segmented organs, that is, bi-level images, contributes to compression performance. Although the performance of well-established techniques is satisfactory, the sensitivity of ABIC to modality type and the advantage of 3D-RLE when the spatial coherence between the adjacent slices are high results with up to 10 times more CR performance.

CONCLUSION

Adaptive and context-based compression strategies allow effective storage and transfer of segmented binary data, which can be used to re-produce visualizations for better teleradiology practices preserving all interaction mechanisms.

Comparison of semi-automatic and deep learning-based automatic methods for liver segmentation in living liver transplant donors

PURPOSE

To compare the accuracy and repeatability of emerging machine learning based (i.e. deep) automatic segmentation algorithms with those of well-established semi-automatic (interactive) methods for determining liver volume in living liver transplant donors at computerized tomography (CT) imaging.

METHODS

A total of 12 (6 semi-, 6 full-automatic) methods are evaluated. The semi-automatic segmentation algorithms are based on both traditional iterative models including watershed, fast marching, region growing, active contours and modern techniques including robust statistical segmenter and super-pixels. These methods entail some sort of interaction mechanism such as placing initialization seeds on images or determining a parameter range. The automatic methods are based on deep learning and they include three framework templates (DeepMedic, NiftyNet and U-Net) the first two of which are applied with default parameter sets and the last two involve adapted novel model designs. For 20 living donors (6 training and 12 test datasets), a group of imaging scientists and radiologists created ground truths by performing manual segmentations on contrast material-enhanced CT images. Each segmentation is evaluated using five metrics (i.e. volume overlap and relative volume errors, average/RMS/maximum symmetrical surface distances). The results are mapped to a scoring system and a final grade is calculated by taking their average. Accuracy and repeatability were evaluated using slice by slice comparisons and volumetric analysis. Diversity and complementarity are observed through heatmaps. Majority voting and Simultaneous Truth and Performance Level Estimation (STAPLE) algorithms are utilized to obtain the fusion of the individual results.

RESULTS

The top four methods are determined to be automatic deep models having 79.63, 79.46 and 77.15 and 74.50 scores. Intra-user score is determined as 95.14. Overall, deep automatic segmentation outperformed interactive techniques on all metrics. The mean volume of liver of ground truth is found to be 1409.93 mL ± 271.28 mL, while it is calculated as 1342.21 mL ± 231.24 mL using automatic and 1201.26 mL ± 258.13 mL using interactive methods, showing higher accuracy and less variation on behalf of automatic methods. The qualitative analysis of segmentation results showed significant diversity and complementarity enabling the idea of using ensembles to obtain superior results. The fusion of automatic methods reached 83.87 with majority voting and 86.20 using STAPLE that are only slightly less than fusion of all methods that achieved 86.70 (majority voting) and 88.74 (STAPLE).

CONCLUSION

Use of the new deep learning based automatic segmentation algorithms substantially increases the accuracy and repeatability for segmentation and volumetric measurements of liver. Fusion of automatic methods based on ensemble approaches exhibits best results almost without any additional time cost due to potential parallel execution of multiple models.