A stochastic multi-agent approach for medical-image segmentation: Application to tumor segmentation in brain MR images

Comparison of semi-automatic and manual segmentation methods for tumor delineation on head and neck squamous cell carcinoma (HNSCC) positron emission tomography (PET) images

Objective. Accurate and reproducible tumor delineation on positron emission tomography (PET) images is required to validate predictive and prognostic models based on PET radiomic features. Manual segmentation of tumors is time-consuming whereas semi-automatic methods are easily implementable and inexpensive. This study assessed the reliability of semi-automatic segmentation methods over manual segmentation for tumor delineation in head and neck squamous cell carcinoma (HNSCC) PET images.Approach. We employed manual and six semi-automatic segmentation methods (just enough interaction (JEI), watershed, grow from seeds (GfS), flood filling (FF), 30% SUVmax and 40%SUVmax threshold) using 3D slicer software to extract 128 radiomic features from FDG-PET images of 100 HNSCC patients independently by three operators. We assessed the distributional properties of all features and considered 92 log-transformed features for subsequent analysis. For each paired comparison of a feature, we fitted a separate linear mixed effect model using the method (two levels; manual versus one semi-automatic method) as a fixed effect and the subject and the operator as the random effects. We estimated different statistics-the intraclass correlation coefficient agreement (aICC), limits of agreement (LoA), total deviation index (TDI), coverage probability (CP) and coefficient of individual agreement (CIA)-to evaluate the agreement between the manual and semi-automatic methods.Main results. Accounting for all statistics across 92 features, the JEI method consistently demonstrated acceptable agreement with the manual method, with median values of aICC = 0.86, TDI = 0.94, CP = 0.66, and CIA = 0.91.Significance. This study demonstrated that JEI method is a reliable semi-automatic method for tumor delineation on HNSCC PET images.

Brain tumor detection based on a novel and high-quality prediction of the tumor pixel distributions

The work presented in this paper is in the area of brain tumor detection. We propose a fast detection system with 3D MRI scans of Flair modality. It performs 2 functions, predicting the gray level distribution and location distribution of the pixels in the tumor regions and generating tumor masks with pixel-wise precision. To facilitate 3D data analysis and processing, we introduce a 2D histogram presentation encompassing the gray-level distribution and pixel-location distribution of a 3D object. In the proposed system, specific 2D histograms highlighting tumor-related features are established by exploiting the left-right asymmetry of a brain structure. A modulation function, generated from the input data of each patient case, is applied to the 2D histograms to transform them into coarsely or finely predicted distributions of tumor pixels. The prediction result helps to identify/remove tumor-free slices. The prediction and removal operations are performed to the axial, coronal and sagittal slice series of a brain image, transforming it into a 3D minimum bounding box of its tumor region. The bounding box is utilized to finalize the prediction and generate a 3D tumor mask. The proposed system has been tested extensively with the data of more than 1200 patient cases in BraTS2018∼2021 datasets. The test results demonstrate that the predicted 2D histograms resemble closely the true ones. The system delivers also very good tumor detection results, comparable to those of state-of-the-art CNN systems with mono-modality inputs. They are reproducible and obtained at an extremely low computation cost and without need for training.

Modern Image-Guided Surgery: A Narrative Review of Medical Image Processing and Visualization

Medical image analysis forms the basis of image-guided surgery (IGS) and many of its fundamental tasks. Driven by the growing number of medical imaging modalities, the research community of medical imaging has developed methods and achieved functionality breakthroughs. However, with the overwhelming pool of information in the literature, it has become increasingly challenging for researchers to extract context-relevant information for specific applications, especially when many widely used methods exist in a variety of versions optimized for their respective application domains. By being further equipped with sophisticated three-dimensional (3D) medical image visualization and digital reality technology, medical experts could enhance their performance capabilities in IGS by multiple folds. The goal of this narrative review is to organize the key components of IGS in the aspects of medical image processing and visualization with a new perspective and insights. The literature search was conducted using mainstream academic search engines with a combination of keywords relevant to the field up until mid-2022. This survey systemically summarizes the basic, mainstream, and state-of-the-art medical image processing methods as well as how visualization technology like augmented/mixed/virtual reality (AR/MR/VR) are enhancing performance in IGS. Further, we hope that this survey will shed some light on the future of IGS in the face of challenges and opportunities for the research directions of medical image processing and visualization.

Automatic segmentation of white matter hyperintensities and correlation analysis for cerebral small vessel disease

Objective

Cerebral white matter hyperintensity can lead to cerebral small vessel disease, MRI images in the brain are used to assess the degree of pathological changes in white matter regions. In this paper, we propose a framework for automatic 3D segmentation of brain white matter hyperintensity based on MRI images to address the problems of low accuracy and segmentation inhomogeneity in 3D segmentation. We explored correlation analyses of cognitive assessment parameters and multiple comparison analyses to investigate differences in brain white matter hyperintensity volume among three cognitive states, Dementia, MCI and NCI. The study explored the correlation between cognitive assessment coefficients and brain white matter hyperintensity volume.

Methods

This paper proposes an automatic 3D segmentation framework for white matter hyperintensity using a deep multi-mapping encoder-decoder structure. The method introduces a 3D residual mapping structure for the encoder and decoder. Multi-layer Cross-connected Residual Mapping Module (MCRCM) is proposed in the encoding stage to enhance the expressiveness of model and perception of detailed features. Spatial Attention Weighted Enhanced Supervision Module (SAWESM) is proposed in the decoding stage to adjust the supervision strategy through a spatial attention weighting mechanism. This helps guide the decoder to perform feature reconstruction and detail recovery more effectively.

Result

Experimental data was obtained from a privately owned independent brain white matter dataset. The results of the automatic 3D segmentation framework showed a higher segmentation accuracy compared to nnunet and nnunet-resnet, with a p-value of <0.001 for the two cognitive assessment parameters MMSE and MoCA. This indicates that larger brain white matter are associated with lower scores of MMSE and MoCA, which in turn indicates poorer cognitive function. The order of volume size of white matter hyperintensity in the three groups of cognitive states is dementia, MCI and NCI, respectively.

Conclusion

The paper proposes an automatic 3D segmentation framework for brain white matter that achieves high-precision segmentation. The experimental results show that larger volumes of segmented regions have a negative correlation with lower scoring coefficients of MMSE and MoCA. This correlation analysis provides promising treatment prospects for the treatment of cerebral small vessel diseases in the brain through 3D segmentation analysis of brain white matter. The differences in the volume of white matter hyperintensity regions in subjects with three different cognitive states can help to better understand the mechanism of cognitive decline in clinical research.

Multi-agent medical image segmentation: A survey

During the last decades, the healthcare area has increasingly relied on medical imaging for the diagnosis of a growing number of pathologies. The different types of medical images are mostly manually processed by human radiologists for diseases detection and monitoring. However, such a procedure is time-consuming and relies on expert judgment. The latter can be influenced by a variety of factors. One of the most complicated image processing tasks is image segmentation. Medical image segmentation consists of dividing the input image into a set of regions of interest, corresponding to body tissues and organs. Recently, artificial intelligence (AI) techniques brought researchers attention with their promising results for the image segmentation automation. Among AI-based techniques are those that use the Multi-Agent System (MAS) paradigm. This paper presents a comparative study of the multi-agent approaches dedicated to the segmentation of medical images, recently published in the literature.

FMTM-feature-map-based transform model for brain image segmentation in tumor detection

The segmentation of brain images is a leading quantitative measure for detecting physiological changes and for analysing structural functions. Based on trends and dimensions of brain, the images indicate heterogeneity. Accurate brain tumour segmentation remains a critical challenge despite the persistent efforts of researchers were owing to a variety of obstacles. This impacts the outcome of tumour detection, causing errors. For addressing this issue, a Feature-Map based Transform Model (FMTM) is introduced to focus on heterogeneous features of input picture to map differences and intensity based on transition Fourier. Unchecked machine learning is used for reliable characteristic map recognition in this mapping process. For the determination of severity and variability, the method of identification depends on symmetry and texture. Learning instances are taught to improve precision using predefined data sets, regardless of loss of labels. The process is recurring until the maximum precision of tumour detection is achieved in low convergence. In this research, FMTM has been applied to brain tumour segmentation to automatically extract feature representations and produce accurate and steady performance because of promising performance made by powerful transition Fourier methods. The suggested model's performance is shown by the metrics processing time, precision, accuracy, and F1-Score.

Habitat Analysis of Breast Cancer-Enhanced MRI Reflects BRCA1 Mutation Determined by Immunohistochemistry

Objective

To use habitat analysis (also termed habitat imaging) for classifying untreated breast cancer-enhanced magnetic resonance imaging (MRI) in women. Moreover, we intended to obtain clustering parameters to predict the BReast CAncer gene 1 (BRCA1) gene mutation and to determine the use of MRI as a noninvasive examination tool.

Methods

We obtained enhanced MRI data of patients with breast cancer before treatment and selected some sequences as the source of habitat imaging. We used the k-means clustering to classify these images. According to the formed subregions, we calculated several parameters to evaluate the clustering. We used immunohistochemistry to detect BRCA1 mutations. Moreover, we separately determined the ability of these parameters through independent modeling or multiple parameter joint modeling to predict these mutations.

Results

Of all extracted values, separation (SP) demonstrated the best prediction performance for a single parameter (area under the receiver operating characteristic curve (AUC), 0.647; 95% confidence interval (CI), 0.557–0.731). Simultaneously, models based on the Calinski-Harabasz Index and sum of square error performed better in the training (AUC, 0.903; 95% CI, 0.831–0.96) and verification (AUC, 0.845; 95% CI, 0.723–0.942) sets for multiparameter joint modeling.

Conclusion

Based on the enhanced MRI of breast tumors and the subregions generated according to the habitat imaging theory, the parameters extracted to describe the clustering effect could reflect the BRCA1 status. Differences between clusters, including the general differences of cluster centers and clusters and the similarity of samples within clusters, were the embodiment of this mutation. We propose an algorithm to predict the BRCA1 mutation of a patient according to the enhanced MRI of the breast tumor.

Secure Collaborative Augmented Reality Framework for Biomedical Informatics

Augmented reality is currently a great interest in biomedical health informatics. At the same time, several challenges have been appeared, in particular with the rapid progress of smart sensors technologies, and medical artificial intelligence. This yields the necessity of new needs in biomedical health informatics. Collaborative learning and privacy are some of the challenges of augmented reality technology in biomedical health informatics. This paper introduces a novel secure collaborative augmented reality framework for biomedical health informatics-based applications. Distributed deep learning is first performed across a multi-agent system platform. The privacy strategy is developed for ensuring better communications of the different intelligent agents in the system. In this research work, a system of multiple agents is created for the simulation of the collective behaviours of the smart components of biomedical health informatics. Augmented reality is also incorporated for better visualization of the resulted medical patterns. A novel privacy strategy based on blockchain is investigated for ensuring the confidentiality of the learning process. Experiments are conducted on the real use case of the biomedical segmentation process. Our strong experimental analysis reveals the strength of the proposed framework when directly compared to state-of-the-art biomedical health informatics solutions.

Machine Learning in Pituitary Surgery

Machine learning applications in neurosurgery are increasingly reported for diverse tasks such as faster and more accurate preoperative diagnosis, enhanced lesion characterization, as well as surgical outcome, complications and healthcare cost prediction. Even though the pertinent literature in pituitary surgery is less extensive with respect to other neurosurgical diseases, past research attempted to answer clinically relevant questions to better assist surgeons and clinicians. In the present chapter we review reported ML applications in pituitary surgery including differential diagnosis, preoperative lesion characterization (immunohistochemistry, cavernous sinus invasion, tumor consistency), surgical outcome and complication predictions (gross total resection, tumor recurrence, and endocrinological remission, cerebrospinal fluid leak, postoperative hyponatremia). Moreover, we briefly discuss from a practical standpoint the current barriers to clinical translation of machine learning research. On the topic of pituitary surgery, published reports can be considered mostly preliminary, requiring larger training populations and strong external validation. Thoughtful selection of clinically relevant outcomes of interest and transversal application of model development pipeline-together with accurate methodological planning and multicenter collaborations-have the potential to overcome current limitations and ultimately provide additional tools for more informed patient management.

Machine Learning Algorithms in Neuroimaging: An Overview

Machine learning (ML) and artificial intelligence (AI) applications in the field of neuroimaging have been on the rise in recent years, and their clinical adoption is increasing worldwide. Deep learning (DL) is a field of ML that can be defined as a set of algorithms enabling a computer to be fed with raw data and progressively discover-through multiple layers of representation-more complex and abstract patterns in large data sets. The combination of ML and radiomics, namely the extraction of features from medical images, has proven valuable, too: Radiomic information can be used for enhanced image characterization and prognosis or outcome prediction. This chapter summarizes the basic concepts underlying ML application for neuroimaging and discusses technical aspects of the most promising algorithms, with a specific focus on Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs), in order to provide the readership with the fundamental theoretical tools to better understand ML in neuroimaging. Applications are highlighted from a practical standpoint in the last section of the chapter, including: image reconstruction and restoration, image synthesis and super-resolution, registration, segmentation, classification, and outcome prediction.

Brain tissue development of neonates with Congenital Septal Defect: Study on MRI Image Evaluation of Deep Learning Algorithm

Objectives:

This article is based on deep learning algorithms and uses MRI to study the development of congenital heart septal defects in neonatal brain tissue.

Methods:

From January 2018 to December 2019, 150 cases of congenital cardiac paper septal defect were retrospectively analyzed on 50 cases of normal newborns and neonates. The four index parametersbrain MR imaging, lateral ventricle pre-angle measurement index (F/F’), body index (D/ D’), caudal nucleus index (C/C’) were analyzed. The independent sample t test is performed to compare the difference parameters between groups.

Results:

F congenital heart disease group and control group/F ‘values were 0.301 ± 0.035 and 0.296 ± 0.031; Evans index was 0.239 ± 0.052 and 0.233 ± 0.025; 2 sets of D/D’ values were 0.261 ± 0.039 and 0.234 ± 0.032; C/C ‘value was 0.138 ± 0.018 and 0.124 ± 0.015 respectively. The congenital heart disease group D/D ‘, and the value of C/C’ obtained under the ROC curve area value, respectively 0.698 and 0.750, Youden index corresponding to the maximum D/D ‘, and the value of C/C’ values were 0.28 and 0.12.

Conclusion:

Lateral ventricle D/D ‘and C/C’ is more sensitive indicator which can be evaluated with the difference between the volume of congenital heart septal defects in newborn normal neonatal brain; when the D/D ‘value> 0.28, C/C’ value> 0.12. For the diagnosis and evaluation of congenital heart septal defect neonatal brain volume abnormalities have a certain reference value.

Magnetic Resonance Imaging Segmentation via Weighted Level Set Model Based on Local Kernel Metric and Spatial Constraint

Magnetic resonance imaging (MRI) segmentation is a fundamental and significant task since it can guide subsequent clinic diagnosis and treatment. However, images are often corrupted by defects such as low-contrast, noise, intensity inhomogeneity, and so on. Therefore, a weighted level set model (WLSM) is proposed in this study to segment inhomogeneous intensity MRI destroyed by noise and weak boundaries. First, in order to segment the intertwined regions of brain tissue accurately, a weighted neighborhood information measure scheme based on local multi information and kernel function is designed. Then, the membership function of fuzzy c-means clustering is used as the spatial constraint of level set model to overcome the sensitivity of level set to initialization, and the evolution of level set function can be adaptively changed according to different tissue information. Finally, the distance regularization term in level set function is replaced by a double potential function to ensure the stability of the energy function in the evolution process. Both real and synthetic MRI images can show the effectiveness and performance of WLSM. In addition, compared with several state-of-the-art models, segmentation accuracy and Jaccard similarity coefficient obtained by WLSM are increased by 0.0586, 0.0362 and 0.1087, 0.0703, respectively.

Efficient brain tumor detection and classification using magnetic resonance imaging

Magnetic Resonance Imaging (MRI) inputs are most noticeable in diagnosing brain tumors via computer and manual clinical understanding. Multi-level detection and classification of the images utilizing computer-aided processing depend on labels and annotations. Though the two processes are dynamic and time-consuming, without which the precise accuracy is less assured. For augmenting the accuracy in processing un-labeled or annotation-less images, this article introduces Absolute Classification-Detection Model (AC-DM). This model uses a conventional neural network for training the morphological variations proficient of achieving label-less classification and tumor detection. The traditional neural network trains the images based on differential lattice morphology for classification and detection. In this process, training for the lattices and their corresponding gradients is validated to improve the precision of the regional analysis. This helps to retain the precision of identifying tumors. The variations are recognized for their lattice mapping in the detected boundaries of the input image. The detected boundaries help to map accurate lattices for adapting morphological transforms. Thus, the partial and complex processing in detecting tumors is restrained in the suggested model, adapting to the classification. The efficiency of the suggested model is verified utilizing accuracy, precision, sensitivity, and classification time.