Image segmentation evaluation: a survey of methods

Evaluation of AI-based nerve segmentation on ultrasound: relevance of standard metrics in the clinical setting

BACKGROUND

In artificial intelligence for ultrasound-guided regional anaesthesia, accurate nerve identification is essential. The technology community typically favours objective metrics of pixel overlap on still-frame images, whereas clinical assessments often use subjective evaluation of cine loops by physician experts. No clinically acceptable threshold of pixel overlap has been defined for nerve segmentation. We investigated the relationship between these approaches and identify thresholds for objective pixel-based metrics when clinical evaluations identify high-quality nerve segmentation.

METHODS

cNerve™ is a deep learning segmentation tool on GE Healthcare's Venue™ ultrasound systems. It highlights nerves of the interscalene-supraclavicular-level brachial plexus, femoral, and popliteal-level sciatic block regions. Expert anaesthesiologists subjectively rated overall segmentation quality of cNerve™ on ultrasound cine loop sequences using a 1-5 Likert scale (1 = poor; 5 = excellent). Objective assessments of nerve segmentation, using the Intersection over Union and Dice similarity coefficient metrics, were applied to frames from sequences rated 5.

RESULTS

A total of 173 still image frames were analysed. The median Intersection over Union for nerves was 0.49, and the median Dice similarity coefficient was 0.65, indicating variable performance based on objective metrics, despite subjective clinical evaluations rating the artificial intelligence-generated nerve segmentation as excellent.

CONCLUSIONS

Variable objective segmentation metric scores correspond to excellent performance on clinically oriented assessment and lack the context provided by subjective expert evaluations. Further work is needed to establish standardised evaluation criteria that incorporate both objective pixel-based and subjective clinical assessments. Collaboration between clinicians and technologists is needed to develop these evaluation methods for improved clinical applicability.

A Comprehensive Survey of Deep Learning Approaches in Image Processing

The integration of deep learning (DL) into image processing has driven transformative advancements, enabling capabilities far beyond the reach of traditional methodologies. This survey offers an in-depth exploration of the DL approaches that have redefined image processing, tracing their evolution from early innovations to the latest state-of-the-art developments. It also analyzes the progression of architectural designs and learning paradigms that have significantly enhanced the ability to process and interpret complex visual data. Key advancements, such as techniques improving model efficiency, generalization, and robustness, are examined, showcasing DL's ability to address increasingly sophisticated image-processing tasks across diverse domains. Metrics used for rigorous model evaluation are also discussed, underscoring the importance of performance assessment in varied application contexts. The impact of DL in image processing is highlighted through its ability to tackle complex challenges and generate actionable insights. Finally, this survey identifies potential future directions, including the integration of emerging technologies like quantum computing and neuromorphic architectures for enhanced efficiency and federated learning for privacy-preserving training. Additionally, it highlights the potential of combining DL with emerging technologies such as edge computing and explainable artificial intelligence (AI) to address scalability and interpretability challenges. These advancements are positioned to further extend the capabilities and applications of DL, driving innovation in image processing.

Mitochondrial segmentation and function prediction in live-cell images with deep learning

Mitochondrial morphology and function are intrinsically linked, indicating the opportunity to predict functions by analyzing morphological features in live-cell imaging. Herein, we introduce MoDL, a deep learning algorithm for mitochondrial image segmentation and function prediction. Trained on a dataset of 20,000 manually labeled mitochondria from super-resolution (SR) images, MoDL achieves superior segmentation accuracy, enabling comprehensive morphological analysis. Furthermore, MoDL predicts mitochondrial functions by employing an ensemble learning strategy, powered by an extended training dataset of over 100,000 SR images, each annotated with functional data from biochemical assays. By leveraging this large dataset alongside data fine-tuning and retraining, MoDL demonstrates the ability to precisely predict functions of heterogeneous mitochondria from unseen cell types through small sample size training. Our results highlight the MoDL's potential to significantly impact mitochondrial research and drug discovery, illustrating its utility in exploring the complex relationship between mitochondrial form and function within a wide range of biological contexts.

TOM500: A Multi-Organ Annotated Orbital MRI Dataset for Thyroid Eye Disease

This study presents TOM500, a comprehensive multi-organ annotated orbital magnetic resonance imaging (MRI) dataset. It includes clinical data, T2-weighted MRI scans, and corresponding segmentations from 500 patients with thyroid eye disease (TED) during their initial visit. TED is a common autoimmune disorder with distinct orbital MRI features. Segmentations of nine orbital structures, including the optic nerve, orbital fat, lacrimal gland, eyeball, and five extraocular muscles (superior rectus and levator palpebrae superioris complex, inferior rectus, medial rectus, lateral rectus, and superior oblique), were generated by three junior annotators and reviewed by an expert radiologist. The consistency of the segmentations was evaluated using the intraclass correlation coefficient. Clinical data, including sex, age, disease duration, and smoking status, are also provided for disease diagnosis and classification. TOM500, the largest publicly available orbital MRI dataset with expert annotations, is designed to facilitate the development of advanced computational tools for TED diagnosis, classification, and observation.

A Multi-Source Circular Geodesic Voting Model for Image Segmentation

Image segmentation is a crucial task in artificial intelligence fields such as computer vision and medical imaging. While convolutional neural networks (CNNs) have achieved notable success by learning representative features from large datasets, they often lack geometric priors and global object information, limiting their accuracy in complex scenarios. Variational methods like active contours provide geometric priors and theoretical interpretability but require manual initialization and are sensitive to hyper-parameters. To overcome these challenges, we propose a novel segmentation approach, named PolarVoting, which combines the minimal path encoding rich geometric features and CNNs which can provide efficient initialization. The introduced model involves two main steps: firstly, we leverage the PolarMask model to extract multiple source points for initialization, and secondly, we construct a voting score map which implicitly contains the segmentation mask via a modified circular geometric voting (CGV) scheme. This map embeds global geometric information for finding accurate segmentation. By integrating neural network representation with geometric priors, the PolarVoting model enhances segmentation accuracy and robustness. Extensive experiments on various datasets demonstrate that the proposed PolarVoting method outperforms both PolarMask and traditional single-source CGV models. It excels in challenging imaging scenarios characterized by intensity inhomogeneity, noise, and complex backgrounds, accurately delineating object boundaries and advancing the state of image segmentation.

A Large-Scale Building Unsupervised Extraction Method Leveraging Airborne LiDAR Point Clouds and Remote Sensing Images Based on a Dual P-Snake Model

Automatic large-scale building extraction from the LiDAR point clouds and remote sensing images is a growing focus in the fields of the sensor applications and remote sensing. However, this building extraction task remains highly challenging due to the complexity of building sizes, shapes, and surrounding environments. In addition, the discreteness, sparsity, and irregular distribution of point clouds, lighting, and shadows, as well as occlusions of the images, also seriously affect the accuracy of building extraction. To address the above issues, we propose a new unsupervised building extraction algorithm PBEA (Point and Pixel Building Extraction Algorithm) based on a new dual P-snake model (Dual Point and Pixel Snake Model). The proposed dual P-snake model is an enhanced active boundary model, which uses both point clouds and images simultaneously to obtain the inner and outer boundaries. The proposed dual P-snake model enables interaction and convergence between the inner and outer boundaries to improve the performance of building boundary detection, especially in complex scenes. Using the dual P-snake model and polygonization, this proposed PBEA can accurately extract large-scale buildings. We evaluated our PBEA and dual P-snake model on the ISPRS Vaihingen dataset and the Toronto dataset. The experimental results show that our PBEA achieves an area-based quality evaluation metric of 90.0% on the Vaihingen dataset and achieves the area-based quality evaluation metric of 92.4% on the Toronto dataset. Compared with other methods, our method demonstrates satisfactory performance.

The edge artifact extraction method for Si3N4 ceramic bearing rolling element microcracks characterization

Aiming at the problem that the edge artifacts of Si3N4 ceramic bearing rolling element microcracks have low contrast, contain noise, and easily merge with the background, making it difficult to segment. A method based on 2D discrete wavelet transform and Otsu threshold segmentation is designed to achieve the extraction of microcrack edge artifact features. Wavelet decomposition is used to remove noise, while wavelet reconstruction features are used to restore lost details. Creation of 2D discrete wavelet transform functional equations combining wavelet reconstruction and wavelet decomposition to improve contrast and eliminate noise in images featuring edge artifacts. For the problem of feature edge artifacts that are difficult to remove, the threshold segmentation function equation is designed to maximize the interclass variance, and the optimal threshold value is selected to remove the edge artifacts. The experimental results show that the average PSNR of the Si3N4 ceramic bearing rolling body point, line, and surface microcrack edge artifact feature images enhanced by the method in this paper is close to 62.69 dB, and the average SSIM is about 0.77. The method in this paper improves the contrast of microcrack edge artifact features of Si3N4 ceramic bearing rolling bodies and makes the feature extraction effect of point, line, and surface microcrack edge artifacts more complete.

USCT-UNet: Rethinking the Semantic Gap in U-Net Network From U-Shaped Skip Connections With Multichannel Fusion Transformer

Medical image segmentation is a crucial component of computer-aided clinical diagnosis, with state-of-the-art models often being variants of U-Net. Despite their success, these models' skip connections introduce an unnecessary semantic gap between the encoder and decoder, which hinders their ability to achieve the high precision required for clinical applications. Awareness of this semantic gap and its detrimental influences have increased over time. However, a quantitative understanding of how this semantic gap compromises accuracy and reliability remains lacking, emphasizing the need for effective mitigation strategies. In response, we present the first quantitative evaluation of the semantic gap between corresponding layers of U-Net and identify two key characteristics: 1) The direct skip connection (DSC) exhibits a semantic gap that negatively impacts models' performance; 2) The magnitude of the semantic gap varies across different layers. Based on these findings, we re-examine this issue through the lens of skip connections. We introduce a Multichannel Fusion Transformer (MCFT) and propose a novel USCT-UNet architecture, which incorporates U-shaped skip connections (USC) to replace DSC, allocates varying numbers of MCFT blocks based on the semantic gap magnitude at different layers, and employs a spatial channel cross-attention (SCCA) module to facilitate the fusion of features between the decoder and USC. We evaluate USCT-UNet on four challenging datasets, and the results demonstrate that it effectively eliminates the semantic gap. Compared to using DSC, our USC and SCCA strategies achieve maximum improvements of 4.79% in the Dice coefficient, 5.70% in mean intersection over union (MIoU), and 3.26 in Hausdorff distance.

Significant effect of image contrast enhancement on weld defect detection

Weld defect inspection is an essential aspect of testing in industries field. From a human viewpoint, a manual inspection can make appropriate justification more difficult and lead to incorrect identification during weld defect detection. Weld defect inspection uses X-radiography testing, which is now mostly outdated. Recently, numerous researchers have utilized X-radiography digital images to inspect the defect. As a result, for error-free inspection, an autonomous weld detection and classification system are required. One of the most difficult issues in the field of image processing, particularly for enhancing image quality, is the issue of contrast variation and luminosity. Enhancement is carried out by adjusting the brightness of the dark or bright intensity to boost segmentation performance and image quality. To equalize contrast variation and luminosity, many different approaches have recently been put forth. In this research, a novel approach called Hybrid Statistical Enhancement (HSE), which is based on a direct strategy using statistical data, is proposed. The HSE method divided each pixel into three groups, the foreground, border, and problematic region, using the mean and standard deviation of a global and local neighborhood (luminosity and contrast). To illustrate the impact of the HSE method on the segmentation or detection stage, the datasets, specifically the weld defect image, were used. Bernsen and Otsu's methods are the two segmentation techniques utilized. The findings from the objective and visual elements demonstrated that the HSE approach might automatically improve segmentation output while effectively enhancing contrast variation and normalizing luminosity. In comparison to the Homomorphic Filter (HF) and Difference of Gaussian (DoG) approaches, the segmentation results for HSE images had the lowest result according to Misclassification Error (ME). After being applied to the HSE images during the segmentation stage, every quantitative result showed an increase. For example, accuracy increased from 64.171 to 84.964. In summary, the application of the HSE method has resulted in an effective and efficient outcome for background correction as well as improving the quality of images.

Application of Recurrence Plot Analysis to Examine Dynamics of Biological Molecules on the Example of Aggregation of Seed Mucilage Components

The goal of the research is to describe the aggregation process inside the mucilage produced by plant seeds using molecular dynamics (MD) combined with time series algorithmic analysis based on the recurrence plots. The studied biological molecules model is seed mucilage composed of three main polysaccharides, i.e. pectins, hemicellulose, and cellulose. The modeling of biological molecules is based on the assumption that a classical-quantum passage underlies the aggregation process in the mucilage, resulting from non-covalent interactions, as they affect the macroscopic properties of the system. The applied recurrence plot approach is an important tool for time series analysis and data mining dedicated to analyzing time series data originating from complex, chaotic systems. In the current research, we demonstrated that advanced algorithmic analysis of seed mucilage data can reveal some features of the dynamics of the system, namely temperature-dependent regions with different dynamics of increments of a number of hydrogen bonds and regions of stable oscillation of increments of a number of hydrophobic-polar interactions. Henceforth, we pave the path for automatic data-mining methods for the analysis of biological molecules with the intermediate step of the application of recurrence plot analysis, as the generalization of recurrence plot applications to other (biological molecules) datasets is straightforward.

Three-Direction Fusion for Accurate Volumetric Liver and Tumor Segmentation

Biomedical image segmentation of organs, tissues and lesions has gained increasing attention in clinical treatment planning and navigation, which involves the exploration of two-dimensional (2D) and three-dimensional (3D) contexts in the biomedical image. Compared to 2D methods, 3D methods pay more attention to inter-slice correlations, which offer additional spatial information for image segmentation. An organ or tumor has a 3D structure that can be observed from three directions. Previous studies focus only on the vertical axis, limiting the understanding of the relationship between a tumor and its surrounding tissues. Important information can also be obtained from sagittal and coronal axes. Therefore, spatial information of organs and tumors can be obtained from three directions, i.e. the sagittal, coronal and vertical axes, to understand better the invasion depth of tumor and its relationship with the surrounding tissues. Moreover, the edges of organs and tumors in biomedical image may be blurred. To address these problems, we propose a three-direction fusion volumetric segmentation (TFVS) model for segmenting 3D biomedical images from three perspectives in sagittal, coronal and transverse planes, respectively. We use the dataset of the liver task provided by the Medical Segmentation Decathlon challenge to train our model. The TFVS method demonstrates a competitive performance on the 3D-IRCADB dataset. In addition, the t-test and Wilcoxon signed-rank test are also performed to show the statistical significance of the improvement by the proposed method as compared with the baseline methods. The proposed method is expected to be beneficial in guiding and facilitating clinical diagnosis and treatment.

Unsupervised local center of mass based scoliosis spinal segmentation and Cobb angle measurement

Scoliosis is a medical condition in which a person's spine has an abnormal curvature and Cobb angle is a measurement used to evaluate the severity of a spinal curvature. Presently, automatic Existing Cobb angle measurement techniques require huge dataset, time-consuming, and needs significant effort. So, it is important to develop an unsupervised method for the measurement of Cobb angle with good accuracy. In this work, an unsupervised local center of mass (LCM) technique is proposed to segment the spine region and further novel Cobb angle measurement method is proposed for accurate measurement. Validation of the proposed method was carried out on 2D X-ray images from the Saudi Arabian population. Segmentation results were compared with GMM-Based Hidden Markov Random Field (GMM-HMRF) segmentation method based on sensitivity, specificity, and dice score. Based on the findings, it can be observed that our proposed segmentation method provides an overall accuracy of 97.3% whereas GMM-HMRF has an accuracy of 89.19%. Also, the proposed method has a higher dice score of 0.54 compared to GMM-HMRF. To further evaluate the effectiveness of the approach in the Cobb angle measurement, the results were compared with Senior Scoliosis Surgeon at Multispecialty Hospital in Saudi Arabia. The findings indicated that the segmentation of the scoliotic spine was nearly flawless, and the Cobb angle measurements obtained through manual examination by the expert and the algorithm were nearly identical, with a discrepancy of only ± 3 degrees. Our proposed method can pave the way for accurate spinal segmentation and Cobb angle measurement among scoliosis patients by reducing observers' variability.

Impact of deep learning-based multiorgan segmentation methods on patient-specific internal dosimetry in PET/CT imaging: A comparative study

PURPOSE

Accurate and fast multiorgan segmentation is essential in image-based internal dosimetry in nuclear medicine. While conventional manual PET image segmentation is widely used, it suffers from both being time-consuming as well as subject to human error. This study exploited 2D and 3D deep learning (DL) models. Key organs in the trunk of the body were segmented and then used as a reference for networks.

METHODS

The pre-trained p2p-U-Net-GAN and HighRes3D architectures were fine-tuned with PET-only images as inputs. Additionally, the HighRes3D model was alternatively trained with PET/CT images. Evaluation metrics such as sensitivity (SEN), specificity (SPC), intersection over union (IoU), and Dice scores were considered to assess the performance of the networks. The impact of DL-assisted PET image segmentation methods was further assessed using the Monte Carlo (MC)-derived S-values to be used for internal dosimetry.

RESULTS

A fair comparison with manual low-dose CT-aided segmentation of the PET images was also conducted. Although both 2D and 3D models performed well, the HighRes3D offers superior performance with Dice scores higher than 0.90. Key evaluation metrics such as SEN, SPC, and IoU vary between 0.89-0.93, 0.98-0.99, and 0.87-0.89 intervals, respectively, indicating the encouraging performance of the models. The percentage differences between the manual and DL segmentation methods in the calculated S-values varied between 0.1% and 6% with a maximum attributed to the stomach.

CONCLUSION

The findings prove while the incorporation of anatomical information provided by the CT data offers superior performance in terms of Dice score, the performance of HighRes3D remains comparable without the extra CT channel. It is concluded that both proposed DL-based methods provide automated and fast segmentation of whole-body PET/CT images with promising evaluation metrics. Between them, the HighRes3D is more pronounced by providing better performance and can therefore be the method of choice for 18F-FDG-PET image segmentation.

Deep learning architectures for diagnosing the severity of apple frog-eye leaf spot disease in complex backgrounds

Introduction

In precision agriculture, accurately diagnosing apple frog-eye leaf spot disease is critical for effective disease management. Traditional methods, predominantly relying on labor-intensive and subjective visual evaluations, are often inefficient and unreliable.

Methods

To tackle these challenges in complex orchard environments, we develop a specialized deep learning architecture. This architecture consists of a two-stage multi-network model. The first stage features an enhanced Pyramid Scene Parsing Network (L-DPNet) with deformable convolutions for improved apple leaf segmentation. The second stage utilizes an improved U-Net (D-UNet), optimized with bilinear upsampling and batch normalization, for precise disease spot segmentation.

Results

Our model sets new benchmarks in performance, achieving a mean Intersection over Union (mIoU) of 91.27% for segmentation of both apple leaves and disease spots, and a mean Pixel Accuracy (mPA) of 94.32%. It also excels in classifying disease severity across five levels, achieving an overall precision of 94.81%.

Discussion

This approach represents a significant advancement in automated disease quantification, enhancing disease management in precision agriculture through data-driven decision-making.

Multi-wavelength interference phase imaging for automatic breast cancer detection and delineation using diffuse reflection imaging

Millions of women globally are impacted by the major health problem of breast cancer (BC). Early detection of BC is critical for successful treatment and improved survival rates. In this study, we provide a progressive approach for BC detection using multi-wavelength interference (MWI) phase imaging based on diffuse reflection hyperspectral (HS) imaging. The proposed findings are based on the measurement of the interference pattern between the blue (446.6 nm) and red (632 nm) wavelengths. We consider implementing a comprehensive image processing and categorization method based on the use of Fast Fourier (FF) transform analysis pertaining to a change in the refractive index between tumor and normal tissue. We observed that cancer growth affects tissue organization dramatically, as seen by persistently increased refractive index variance in tumors compared normal areas. Both malignant and normal tissue had different depth data collected from it that was analyzed. To enhance the categorization of ex-vivo BC tissue, we developed and validated a training classifier algorithm specifically designed for categorizing HS cube data. Following the application of signal normalization with the FF transform algorithm, our methodology achieved a high level of performance with a specificity (Spec) of 94% and a sensitivity (Sen) of 90.9% for the 632 nm acquired image categorization, based on preliminary findings from breast specimens under investigation. Notably, we successfully leveraged unstained tissue samples to create 3D phase-resolved images that effectively highlight the distinctions in diffuse reflectance features between cancerous and healthy tissue. Preliminary data revealed that our imaging method might be able to assist specialists in safely excising malignant areas and assessing the tumor bed following resection automatically at different depths. This preliminary investigation might result in an effective "in-vivo" disease description utilizing optical technology using a typical RGB camera with wavelength-specific operation with our quantitative phase MWI imaging methodology.

Sediment core analysis using artificial intelligence

Subsurface stratigraphic modeling is crucial for a variety of environmental, societal, and economic challenges. However, the need for specific sedimentological skills in sediment core analysis may constitute a limitation. Methods based on Machine Learning and Deep Learning can play a central role in automatizing this time-consuming procedure. In this work, using a robust dataset of high-resolution digital images from continuous sediment cores of Holocene age that reflect a wide spectrum of continental to shallow-marine depositional environments, we outline a novel deep-learning-based approach to perform automatic semantic segmentation directly on core images, leveraging the power of convolutional neural networks. To optimize the interpretation process and maximize scientific value, we use six sedimentary facies associations as target classes in lieu of ineffective classification methods based uniquely on lithology. We propose an automated model that can rapidly characterize sediment cores, allowing immediate guidance for stratigraphic correlation and subsurface reconstructions.

Influence of Data Augmentation Strategies on the Segmentation of Oral Histological Images Using Fully Convolutional Neural Networks

Segmentation of tumor regions in H &E-stained slides is an important task for a pathologist while diagnosing different types of cancer, including oral squamous cell carcinoma (OSCC). Histological image segmentation is often constrained by the availability of labeled training data since labeling histological images is a highly skilled, complex, and time-consuming task. Thus, data augmentation strategies become essential to train convolutional neural networks models to overcome the overfitting problem when only a few training samples are available. This paper proposes a new data augmentation strategy, named Random Composition Augmentation (RCAug), to train fully convolutional networks (FCN) to segment OSCC tumor regions in H &E-stained histological images. Given the input image and their corresponding label, a pipeline with a random composition of geometric, distortion, color transfer, and generative image transformations is executed on the fly. Experimental evaluations were performed using an FCN-based method to segment OSCC regions through a set of different data augmentation transformations. By using RCAug, we improved the FCN-based segmentation method from 0.51 to 0.81 of intersection-over-union (IOU) in a whole slide image dataset and from 0.65 to 0.69 of IOU in a tissue microarray images dataset.

Deep reinforcement learning enables adaptive-image augmentation for automated optical inspection of plant rust

This study proposes an adaptive image augmentation scheme using deep reinforcement learning (DRL) to improve the performance of a deep learning-based automated optical inspection system. The study addresses the challenge of inconsistency in the performance of single image augmentation methods. It introduces a DRL algorithm, DQN, to select the most suitable augmentation method for each image. The proposed approach extracts geometric and pixel indicators to form states, and uses DeepLab-v3+ model to verify the augmented images and generate rewards. Image augmentation methods are treated as actions, and the DQN algorithm selects the best methods based on the images and segmentation model. The study demonstrates that the proposed framework outperforms any single image augmentation method and achieves better segmentation performance than other semantic segmentation models. The framework has practical implications for developing more accurate and robust automated optical inspection systems, critical for ensuring product quality in various industries. Future research can explore the generalizability and scalability of the proposed framework to other domains and applications. The code for this application is uploaded at https://github.com/lynnkobe/Adaptive-Image-Augmentation.git.

φ-OTDR signal compression scheme based on the compressed sensing theory

In this paper, based on the compressed sensing theory and the orthogonal matching pursuit algorithm, we have designed a data compression scheme, taking the Space-Temporal graph, time domain curve, and its time-frequency spectrum of phase-sensitive optical time-domain reflectometer as the target signals. The compression rates of the three signals were 40%, 35%, and 20%, while the average reconstruction times were 0.74 s, 0.49 s, and 0.32 s. The reconstructed samples effectively retained the characteristic blocks, response pulses, and energy distribution that symbolize the presence of vibrations. The average correlation coefficients of the three kinds of reconstructed signals with the original samples were 0.88, 0.85, and 0.86, respectively, and then a series of quantitative metrics were designed to evaluate the reconstructing efficiency. We have utilized the neural network trained by the original data to identify the reconstructed samples with an accuracy of over 70%, indicating that the reconstructed samples accurately present the vibration characteristics.

Performance evaluation of computer-aided automated master frame selection techniques for fetal echocardiography

PURPOSE

Fetal echocardiography is widely used for the assessment of fetal heart development and detection of congenital heart disease (CHD). Preliminary examination of the fetal heart involves the four-chamber view which indicates the presence of all the four chambers and its structural symmetry. Examination of various cardiac parameters is generally done using the clinically selected diastole frame. This largely depends on the expertise of the sonographer and is prone to intra- and interobservational errors. To overcome this, automated frame selection technique is proposed for the recognition of fetal cardiac chamber from fetal echocardiography.

METHODS

Three techniques have been proposed in this research study to automate the process of determining the frame referred as "Master Frame" that can be used for the measurement of the cardiac parameters. The first method uses frame similarity measures (FSM) for the determination of the master frame from the given cine loop ultrasonic sequences. FSM makes use of similarity measures such as correlation, structural similarity index (SSIM), peak signal to noise ratio (PSNR), and mean square error (MSE) to identify the cardiac cycle, and all the frames in one cardiac cycle are superimposed to form the master frame. The final master frame is obtained by considering the average of the master frame obtained using each similarity measure. The second method uses averaging of ± 20% from the midframes (AMF). The third method uses averaging of all the frames (AAF) of the cine loop sequence. Both diastole and master frames have been annotated by the clinical experts, and their ground truths are compared for validation. No segmentation techniques have been used to avoid the variability of the performance of various segmentation techniques. All the proposed schemes were evaluated using six fidelity metrics such as Dice coefficient, Jaccard ratio, Hausdorff distance, structural similarity index, mean absolute error, and Pratt figure of merit.

RESULTS

The three proposed techniques were tested on the frames extracted from 95 ultrasound cine loop sequences between 19 and 32 weeks of gestation. The feasibility of the techniques was determined by the computation of fidelity metrics between the master frame derived and the diastole frame chosen by the clinical experts. The FSM-based identified master frame found to closely match with manually chosen diastole frame and also ensures statistically significant. The method also detects automatically the cardiac cycle. The resultant master frame obtained through AMF though found to be identical to that of the diastole frame, the size of the chambers found to be reduced that can lead to inaccurate chamber measurement. The master frame obtained through AAF was not found to be identical to that of clinical diastole frame.

CONCLUSION

It can be concluded that the frame similarity measure (FSM)-based master frame can be introduced in the clinical routine for segmentation followed by cardiac chamber measurements. Such automated master frame selection also overcomes the manual intervention of earlier reported techniques in the literature. The fidelity metrics assessment further confirms the suitability of proposed master frame for automated fetal chamber recognition.

A novel vessel segmentation algorithm for pathological en-face images based on matched filter

The vascular information in fundus images can provide important basis for detection and prediction of retina-related diseases. However, the presence of lesions such as Coroidal Neovascularization can seriously interfere with normal vascular areas in optical coherence tomography (OCT) fundus images. In this paper, a novel method is proposed for detecting blood vessels in pathological OCT fundus images. First of all, an automatic localization and filling method is used in preprocessing step to reduce pathological interference. Afterwards, in terms of vessel extraction, a pore ablation method based on capillary bundle model is applied. The ablation method processes the image after matched filter feature extraction, which can eliminate the interference caused by diseased blood vessels to a great extent. At the end of the proposed method, morphological operations are used to obtain the main vascular features. Experimental results on the dataset show that the proposed method achieves 0.88 ± 0.03, 0.79 ± 0.05, 0.66 ± 0.04, results in DICE, PRECISION and TPR, respectively. Effective extraction of vascular information from OCT fundus images is of great significance for the diagnosis and treatment of retinal related diseases.

POSEA: A novel algorithm to evaluate the performance of multi-object instance image segmentation

Many techniques and software packages have been developed to segment individual cells within microscopy images, necessitating a robust method to evaluate images segmented into a large number of unique objects. Currently, segmented images are often compared with ground-truth images at a pixel level; however, this standard pixel-level approach fails to compute errors due to pixels incorrectly assigned to adjacent objects. Here, we define a per-object segmentation evaluation algorithm (POSEA) that calculates segmentation accuracy metrics for each segmented object relative to a ground truth segmented image. To demonstrate the performance of POSEA, precision, recall, and f-measure metrics are computed and compared with the standard pixel-level evaluation for simulated images and segmented fluorescence microscopy images of three different cell samples. POSEA yields lower accuracy metrics than the standard pixel-level evaluation due to correct accounting of misclassified pixels of adjacent objects. Therefore, POSEA provides accurate evaluation metrics for objects with pixels incorrectly assigned to adjacent objects and is robust for use across a variety of applications that require evaluation of the segmentation of unique adjacent objects.

Expanding the horizon for breast cancer screening in India through artificial intelligent technologies -A mini-review

Introduction

Breast cancer is one of the most common cancer among Indian women, with an incidence of 25.8 per 100,000 women according to the Ministry of Health and Family Welfare. Late detection is responsible for poor quality of life (QOL), and it is the leading cause of death. In metropolitan regions, one in every 22 women will have breast cancer over their lifetime; but in rural areas, one in every 60 women will develop breast cancer as per estimates.

Aim and objective

This paper aims to describe the various AI based breast screening technologies which are used in breast cancer screening in India.

Methodology

The literature search was done using "Pub Med," "Google scholar," and "Scopus" databases for the key terms "technology," "cancer research," "artificial intelligence," "mammography", "breast cancer," "cancer," and/or "neoplasia in breast." All the relevant articles were included to support this mini review.

Results

We found that emerging artificial intelligent technologies namely "Niramai", "iBreastExam," "MammoAssist" are emerging as an hope for early detection by screening in resource poor settings, in turn, which can improve the QOL among breast cancer patients.

Optimized method for segmentation of ancient mural images based on superpixel algorithm

High-precision segmentation of ancient mural images is the foundation of their digital virtual restoration. However, the complexity of the color appearance of ancient murals makes it difficult to achieve high-precision segmentation when using traditional algorithms directly. To address the current challenges in ancient mural image segmentation, an optimized method based on a superpixel algorithm is proposed in this study. First, the simple linear iterative clustering (SLIC) algorithm is applied to the input mural images to obtain superpixels. Then, the density-based spatial clustering of applications with noise (DBSCAN) algorithm is used to cluster the superpixels to obtain the initial clustered images. Subsequently, a series of optimized strategies, including (1) merging the small noise superpixels, (2) segmenting and merging the large noise superpixels, (3) merging initial clusters based on color similarity and positional adjacency to obtain the merged regions, and (4) segmenting and merging the color-mixing noisy superpixels in each of the merged regions, are applied to the initial cluster images sequentially. Finally, the optimized segmentation results are obtained. The proposed method is tested and compared with existing methods based on simulated and real mural images. The results show that the proposed method is effective and outperforms the existing methods.

A deep learning model for CT-based kidney volume determination in dogs and normal reference definition

Kidney volume is associated with renal function and the severity of renal diseases, thus accurate assessment of the kidney is important. Although the voxel count method is reported to be more accurate than several methods, its laborious and time-consuming process is considered as a main limitation. In need of a new technology that is fast and as accurate as the manual voxel count method, the aim of this study was to develop the first deep learning model for automatic kidney detection and volume estimation from computed tomography (CT) images of dogs. A total of 182,974 image slices from 386 CT scans of 211 dogs were used to develop this deep learning model. Owing to the variance of kidney size and location in dogs compared to humans, several processing methods and an architecture based on UNEt Transformers which is known to show promising results for various medical image segmentation tasks including this study. Combined loss function and data augmentation were applied to elevate the performance of the model. The Dice similarity coefficient (DSC) which shows the similarity between manual segmentation and automated segmentation by deep-learning model was 0.915 ± 0.054 (mean ± SD) with post-processing. Kidney volume agreement analysis assessing the similarity between the kidney volume estimated by manual voxel count method and the deep-learning model was r = 0.960 (p < 0.001), 0.95 from Lin's concordance correlation coefficient (CCC), and 0.975 from the intraclass correlation coefficient (ICC). Kidney volume was positively correlated with body weight (BW), and insignificantly correlated with body conditions score (BCS), age, and sex. The correlations between BW, BCS, and kidney volume were as follows: kidney volume = 3.701 × BW + 11.962 (R 2 = 0.74, p < 0.001) and kidney volume = 19.823 × BW/BCS index + 10.705 (R 2 = 0.72, p < 0.001). The deep learning model developed in this study is useful for the automatic estimation of kidney volume. Furthermore, a reference range established in this study for CT-based normal kidney volume considering BW and BCS can be helpful in assessment of kidney in dogs.

DMDF-Net: Dual multiscale dilated fusion network for accurate segmentation of lesions related to COVID-19 in lung radiographic scans

The recent disaster of COVID-19 has brought the whole world to the verge of devastation because of its highly transmissible nature. In this pandemic, radiographic imaging modalities, particularly, computed tomography (CT), have shown remarkable performance for the effective diagnosis of this virus. However, the diagnostic assessment of CT data is a human-dependent process that requires sufficient time by expert radiologists. Recent developments in artificial intelligence have substituted several personal diagnostic procedures with computer-aided diagnosis (CAD) methods that can make an effective diagnosis, even in real time. In response to COVID-19, various CAD methods have been developed in the literature, which can detect and localize infectious regions in chest CT images. However, most existing methods do not provide cross-data analysis, which is an essential measure for assessing the generality of a CAD method. A few studies have performed cross-data analysis in their methods. Nevertheless, these methods show limited results in real-world scenarios without addressing generality issues. Therefore, in this study, we attempt to address generality issues and propose a deep learning-based CAD solution for the diagnosis of COVID-19 lesions from chest CT images. We propose a dual multiscale dilated fusion network (DMDF-Net) for the robust segmentation of small lesions in a given CT image. The proposed network mainly utilizes the strength of multiscale deep features fusion inside the encoder and decoder modules in a mutually beneficial manner to achieve superior segmentation performance. Additional pre- and post-processing steps are introduced in the proposed method to address the generality issues and further improve the diagnostic performance. Mainly, the concept of post-region of interest (ROI) fusion is introduced in the post-processing step, which reduces the number of false-positives and provides a way to accurately quantify the infected area of lung. Consequently, the proposed framework outperforms various state-of-the-art methods by accomplishing superior infection segmentation results with an average Dice similarity coefficient of 75.7%, Intersection over Union of 67.22%, Average Precision of 69.92%, Sensitivity of 72.78%, Specificity of 99.79%, Enhance-Alignment Measure of 91.11%, and Mean Absolute Error of 0.026.

Survey of Supervised Learning for Medical Image Processing

Medical image interpretation is an essential task for the correct diagnosis of many diseases. Pathologists, radiologists, physicians, and researchers rely heavily on medical images to perform diagnoses and develop new treatments. However, manual medical image analysis is tedious and time consuming, making it necessary to identify accurate automated methods. Deep learning-especially supervised deep learning-shows impressive performance in the classification, detection, and segmentation of medical images and has proven comparable in ability to humans. This survey aims to help researchers and practitioners of medical image analysis understand the key concepts and algorithms of supervised learning techniques. Specifically, this survey explains the performance metrics of supervised learning methods; summarizes the available medical datasets; studies the state-of-the-art supervised learning architectures for medical imaging processing, including convolutional neural networks (CNNs) and their corresponding algorithms, region-based CNNs and their variants, fully convolutional networks (FCN) and U-Net architecture; and discusses the trends and challenges in the application of supervised learning methods to medical image analysis. Supervised learning requires large labeled datasets to learn and achieve good performance, and data augmentation, transfer learning, and dropout techniques have widely been employed in medical image processing to overcome the lack of such datasets.

SinGAN-Seg: Synthetic training data generation for medical image segmentation

Analyzing medical data to find abnormalities is a time-consuming and costly task, particularly for rare abnormalities, requiring tremendous efforts from medical experts. Therefore, artificial intelligence has become a popular tool for the automatic processing of medical data, acting as a supportive tool for doctors. However, the machine learning models used to build these tools are highly dependent on the data used to train them. Large amounts of data can be difficult to obtain in medicine due to privacy reasons, expensive and time-consuming annotations, and a general lack of data samples for infrequent lesions. In this study, we present a novel synthetic data generation pipeline, called SinGAN-Seg, to produce synthetic medical images with corresponding masks using a single training image. Our method is different from the traditional generative adversarial networks (GANs) because our model needs only a single image and the corresponding ground truth to train. We also show that the synthetic data generation pipeline can be used to produce alternative artificial segmentation datasets with corresponding ground truth masks when real datasets are not allowed to share. The pipeline is evaluated using qualitative and quantitative comparisons between real data and synthetic data to show that the style transfer technique used in our pipeline significantly improves the quality of the generated data and our method is better than other state-of-the-art GANs to prepare synthetic images when the size of training datasets are limited. By training UNet++ using both real data and the synthetic data generated from the SinGAN-Seg pipeline, we show that the models trained on synthetic data have very close performances to those trained on real data when both datasets have a considerable amount of training data. In contrast, we show that synthetic data generated from the SinGAN-Seg pipeline improves the performance of segmentation models when training datasets do not have a considerable amount of data. All experiments were performed using an open dataset and the code is publicly available on GitHub.

Challenges and opportunities of integrating imaging and mathematical modelling to interrogate biological processes

Advances in biological imaging have accelerated our understanding of human physiology in both health and disease. As these advances have developed, the opportunities gained by integrating with cutting-edge mathematical models have become apparent yet remain challenging. Combined imaging-modelling approaches provide unprecedented opportunity to correlate data on tissue architecture and function, across length and time scales, to better understand the mechanisms that underpin fundamental biology and also to inform clinical decisions. Here we discuss the opportunities and challenges of such approaches, providing literature examples across a range of organ systems. Given the breadth of the field we focus on the intersection of continuum modelling and in vivo imaging applied to the vasculature and blood flow, though our rationale and conclusions extend widely. We propose three key research pillars (image acquisition, image processing, mathematical modelling) and present their respective advances as well as future opportunity via better integration. Multidisciplinary efforts that develop imaging and modelling tools concurrently, and share them open-source with the research community, provide exciting opportunity for advancing these fields.

An Intelligent Solution for Automatic Garment Measurement Using Image Recognition Technologies

Global digitization trends and the application of high technology in the garment market are still too slow to integrate, despite the increasing demand for automated solutions. The main challenge is related to the extraction of garment information-general clothing descriptions and automatic dimensional extraction. In this paper, we propose the garment measurement solution based on image processing technologies, which is divided into two phases, garment segmentation and key points extraction. UNet as a backbone network has been used for mask retrieval. Separate algorithms have been developed to identify both general and specific garment key points from which the dimensions of the garment can be calculated by determining the distances between them. Using this approach, we have resulted in an average 1.27 cm measurement error for the prediction of the basic measurements of blazers, 0.747 cm for dresses and 1.012 cm for skirts.

Image Segmentation under the Optimization Algorithm of Krill Swarm and Machine Learning

This study aims to improve the efficiency and accuracy of image segmentation, and to compare and study traditional threshold-based image segmentation methods and machine learning model-based image segmentation methods. The krill herb optimization algorithm is combined with the traditional maximum between-class variance function to form a new graph segmentation algorithm. The pet dataset is used to train the algorithm model and build an image semantic segmentation system. The results show that when the traditional Ostu algorithm performs image single-threshold segmentation, the number of iterations is about 256. When double-threshold segmentation is performed, the number of iterations increases exponentially, and the execution time is about 2 s. The number of iterations of the improved Krill Herd algorithm in single-threshold segmentation is 6.95 times, respectively. The execution time for double-threshold segmentation is about 0.24 s. The number of iterations is only improved by a factor of 0.19. The average classification accuracy of the Unet network model and the SegNet network model is 86.3% and 91.9%, respectively. The average classification accuracy of the DC-Unet network model reaches 93.1%. This shows that the proposed fusion algorithm has high optimization efficiency and stronger practicability in multithreshold image segmentation. The DC-Unet network model can improve the image detail segmentation effect. The research provides a new idea for finding an efficient and accurate image segmentation method.

Operational use of multispectral images for macro-litter mapping and categorization by Unmanned Aerial Vehicle

The use of Unmanned Aerial Systems (UAS, aka drones) has shown to be feasible to perform marine litter surveys. We operationally tested the use of multispectral images (5 bands) to classify litter type and material on a beach-dune system. For litter categorization by their multispectral characteristics, the Spectral Angle Mapping (SAM) technique was adopted. The SAM-based categorization of litter agreed with the visual classification, thus multispectral images can be used to fasten and/or making more robust the manual RGB image screening. Fully automated detection returned an F-score of 0.64, and a reasonable categorization of litter. Overall, the image-based litter density maps were in line with the manual detection. Assessments were promising given the complexity of the study area, where different dunes plants and partially-buried items challenged the UAS-based litter detection. The method can be easily implemented for both floating and beached litter, to advance litter survey in the environment.

In vitro machine learning-based CAR T immunological synapse quality measurements correlate with patient clinical outcomes

The human immune system consists of a highly intelligent network of billions of independent, self-organized cells that interact with each other. Machine learning (ML) is an artificial intelligence (AI) tool that automatically processes huge amounts of image data. Immunotherapies have revolutionized the treatment of blood cancer. Specifically, one such therapy involves engineering immune cells to express chimeric antigen receptors (CAR), which combine tumor antigen specificity with immune cell activation in a single receptor. To improve their efficacy and expand their applicability to solid tumors, scientists optimize different CARs with different modifications. However, predicting and ranking the efficacy of different "off-the-shelf" immune products (e.g., CAR or Bispecific T-cell Engager [BiTE]) and selection of clinical responders are challenging in clinical practice. Meanwhile, identifying the optimal CAR construct for a researcher to further develop a potential clinical application is limited by the current, time-consuming, costly, and labor-intensive conventional tools used to evaluate efficacy. Particularly, more than 30 years of immunological synapse (IS) research data demonstrate that T cell efficacy is not only controlled by the specificity and avidity of the tumor antigen and T cell interaction, but also it depends on a collective process, involving multiple adhesion and regulatory molecules, as well as tumor microenvironment, spatially and temporally organized at the IS formed by cytotoxic T lymphocytes (CTL) and natural killer (NK) cells. The optimal function of cytotoxic lymphocytes (including CTL and NK) depends on IS quality. Recognizing the inadequacy of conventional tools and the importance of IS in immune cell functions, we investigate a new strategy for assessing CAR-T efficacy by quantifying CAR IS quality using the glass-support planar lipid bilayer system combined with ML-based data analysis. Previous studies in our group show that CAR-T IS quality correlates with antitumor activities in vitro and in vivo. However, current manually quantified IS quality data analysis is time-consuming and labor-intensive with low accuracy, reproducibility, and repeatability. In this study, we develop a novel ML-based method to quantify thousands of CAR cell IS images with enhanced accuracy and speed. Specifically, we used artificial neural networks (ANN) to incorporate object detection into segmentation. The proposed ANN model extracts the most useful information to differentiate different IS datasets. The network output is flexible and produces bounding boxes, instance segmentation, contour outlines (borders), intensities of the borders, and segmentations without borders. Based on requirements, one or a combination of this information is used in statistical analysis. The ML-based automated algorithm quantified CAR-T IS data correlates with the clinical responder and non-responder treated with Kappa-CAR-T cells directly from patients. The results suggest that CAR cell IS quality can be used as a potential composite biomarker and correlates with antitumor activities in patients, which is sufficiently discriminative to further test the CAR IS quality as a clinical biomarker to predict response to CAR immunotherapy in cancer. For translational research, the method developed here can also provide guidelines for designing and optimizing numerous CAR constructs for potential clinical development. Trial Registration: ClinicalTrials.gov NCT00881920.

Kidney Tumor Semantic Segmentation Using Deep Learning: A Survey of State-of-the-Art

Cure rates for kidney cancer vary according to stage and grade; hence, accurate diagnostic procedures for early detection and diagnosis are crucial. Some difficulties with manual segmentation have necessitated the use of deep learning models to assist clinicians in effectively recognizing and segmenting tumors. Deep learning (DL), particularly convolutional neural networks, has produced outstanding success in classifying and segmenting images. Simultaneously, researchers in the field of medical image segmentation employ DL approaches to solve problems such as tumor segmentation, cell segmentation, and organ segmentation. Segmentation of tumors semantically is critical in radiation and therapeutic practice. This article discusses current advances in kidney tumor segmentation systems based on DL. We discuss the various types of medical images and segmentation techniques and the assessment criteria for segmentation outcomes in kidney tumor segmentation, highlighting their building blocks and various strategies.

A 6D Pose Estimation for Robotic Bin-Picking Using Point-Pair Features with Curvature (Cur-PPF)

Pose estimation is a particularly important link in the task of robotic bin-picking. Its purpose is to obtain the 6D pose (3D position and 3D posture) of the target object. In real bin-picking scenarios, noise, overlap, and occlusion affect accuracy of pose estimation and lead to failure in robot grasping. In this paper, a new point-pair feature (PPF) descriptor is proposed, in which curvature information of point-pairs is introduced to strengthen feature description, and improves the point cloud matching rate. The proposed method also introduces an effective point cloud preprocessing, which extracts candidate targets in complex scenarios, and, thus, improves the overall computational efficiency. By combining with the curvature distribution, a weighted voting scheme is presented to further improve the accuracy of pose estimation. The experimental results performed on public data set and real scenarios show that the accuracy of the proposed method is much higher than that of the existing PPF method, and it is more efficient than the PPF method. The proposed method can be used for robotic bin-picking in real industrial scenarios.

Unknown Object Detection Using a One-Class Support Vector Machine for a Cloud-Robot System

Inter-robot communication and high computational power are challenging issues for deploying indoor mobile robot applications with sensor data processing. Thus, this paper presents an efficient cloud-based multirobot framework with inter-robot communication and high computational power to deploy autonomous mobile robots for indoor applications. Deployment of usable indoor service robots requires uninterrupted movement and enhanced robot vision with a robust classification of objects and obstacles using vision sensor data in the indoor environment. However, state-of-the-art methods face degraded indoor object and obstacle recognition for multiobject vision frames and unknown objects in complex and dynamic environments. From these points of view, this paper proposes a new object segmentation model to separate objects from a multiobject robotic view-frame. In addition, we present a support vector data description (SVDD)-based one-class support vector machine for detecting unknown objects in an outlier detection fashion for the classification model. A cloud-based convolutional neural network (CNN) model with a SoftMax classifier is used for training and identification of objects in the environment, and an incremental learning method is introduced for adding unknown objects to the robot knowledge. A cloud-robot architecture is implemented using a Node-RED environment to validate the proposed model. A benchmarked object image dataset from an open resource repository and images captured from the lab environment were used to train the models. The proposed model showed good object detection and identification results. The performance of the model was compared with three state-of-the-art models and was found to outperform them. Moreover, the usability of the proposed system was enhanced by the unknown object detection, incremental learning, and cloud-based framework.

Preprocessing Effects on Performance of Skin Lesion Saliency Segmentation

Despite the recent advances in immune therapies, melanoma remains one of the deadliest and most difficult skin cancers to treat. Literature reports that multifarious driver oncogenes with tumor suppressor genes are responsible for melanoma progression and its complexity can be demonstrated by alterations in expression with signaling cascades. However, a further improvement in the therapeutic outcomes of the disease is highly anticipated with the aid of humanoid assistive technologies that are nowadays touted as a superlative alternative for the clinical diagnosis of diseases. The development of the projected technology-assistive diagnostics will be based on the innovations of medical imaging, artificial intelligence, and humanoid robots. Segmentation of skin lesions in dermoscopic images is an important requisite component of such a breakthrough innovation for an accurate melanoma diagnosis. However, most of the existing segmentation methods tend to perform poorly on dermoscopic images with undesirable heterogeneous properties. Novel image segmentation methods are aimed to address these undesirable heterogeneous properties of skin lesions with the help of image preprocessing methods. Nevertheless, these methods come with the extra cost of computational complexity and their performances are highly dependent on the preprocessing methods used to alleviate the deteriorating effects of the inherent artifacts. The overarching objective of this study is to investigate the effects of image preprocessing on the performance of a saliency segmentation method for skin lesions. The resulting method from the collaboration of color histogram clustering with Otsu thresholding is applied to demonstrate that preprocessing can be abolished in the saliency segmentation of skin lesions in dermoscopic images with heterogeneous properties. The color histogram clustering is used to automatically determine the initial clusters that represent homogenous regions in an input image. Subsequently, a saliency map is computed by agglutinating color contrast, contrast ratio, spatial feature, and central prior to efficiently detect regions of skin lesions in dermoscopic images. The final stage of the segmentation process is accomplished by applying Otsu thresholding followed by morphological analysis to obliterate the undesirable artifacts that may be present at the saliency detection stage. Extensive experiments were conducted on the available benchmarking datasets to validate the performance of the segmentation method. Experimental results generally indicate that it is passable to segment skin lesions in dermoscopic images without preprocessing because the applied segmentation method is ferociously competitive with each of the numerous leading supervised and unsupervised segmentation methods investigated in this study.

Using Deep Convolutional Neural Network for Image-Based Diagnosis of Nutrient Deficiencies in Plants Grown in Aquaponics

In the aquaponic system, plant nutrients bioavailable from fish excreta are not sufficient for optimal plant growth. Accurate and timely monitoring of the plant’s nutrient status grown in aquaponics is a challenge in order to maintain the balance and sustainability of the system. This study aimed to integrate color imaging and deep convolutional neural networks (DCNNs) to diagnose the nutrient status of lettuce grown in aquaponics. Our approach consists of multi-stage procedures, including plant object detection and classification of nutrient deficiency. The robustness and diagnostic capability of proposed approaches were evaluated using a total number of 3000 lettuce images that were classified into four nutritional classes—namely, full nutrition (FN), nitrogen deficiency (N), phosphorous deficiency (P), and potassium deficiency (K). The performance of the DCNNs was compared with traditional machine learning (ML) algorithms (i.e., Simple thresholding, K-means, support vector machine; SVM, k-nearest neighbor; KNN, and decision Tree; DT). The results demonstrated that the deep proposed segmentation model obtained an accuracy of 99.1%. Also, the deep proposed classification model achieved the highest accuracy of 96.5%. These results indicate that deep learning models, combined with color imaging, provide a promising approach to timely monitor nutrient status of the plants grown in aquaponics, which allows for taking preventive measures and mitigating economic and production losses. These approaches can be integrated into embedded devices to control nutrient cycles in aquaponics.

Photometric Stereo-Based Defect Detection System for Steel Components Manufacturing Using a Deep Segmentation Network

This paper presents an automatic system for the quality control of metallic components using a photometric stereo-based sensor and a customized semantic segmentation network. This system is designed based on interoperable modules, and allows capturing the knowledge of the operators to apply it later in automatic defect detection. A salient contribution is the compact representation of the surface information achieved by combining photometric stereo images into a RGB image that is fed to a convolutional segmentation network trained for surface defect detection. We demonstrate the advantage of this compact surface imaging representation over the use of each photometric imaging source of information in isolation. An empirical analysis of the performance of the segmentation network on imaging samples of materials with diverse surface reflectance properties is carried out, achieving Dice performance index values above 0.83 in all cases. The results support the potential of photometric stereo in conjunction with our semantic segmentation network.

Master Frame Extraction of Fetal Cardiac Images Using B Mode Ultrasound Images

Fetal Echocardiography is used for monitoring the fetal heart and for detection of Congenital Heart Disease (CHD). It is well known that fetal cardiac four chamber view has been widely used for preliminary examination for the detection of CHD. The end diastole frame is generally used for the analysis of the fetal cardiac chambers which is manually picked by the clinician during examination/screening. This method is subjected to intra and inter observer errors and also time consuming. The proposed study aims to automate this process by determining the frame, referred to as the Master frame from the cine loop sequences that can be used for the analysis of the fetal heart chambers instead of the clinically chosen diastole frame. The proposed framework determines the correlation between the reference (first) frame with the successive frames to identify one cardiac cycle. Then the Master frame is formed by superimposing all the frames belonging to one cardiac cycle. The master frame is then compared with the clinically chosen diastole frame in terms of fidelity metrics such as Dice coefficient, Hausdorff distance, mean square error and structural similarity index. The average value of the fidelity metrics considering the dataset used for this study 0.73 for Dice, 13.94 for Hausdorff distance, 0.99 for Structural Similarity Index and 0.035 for mean square error confirms the suitability of the proposed master frame extraction thereby avoiding manual intervention by the clinician. .

Applying an adaptive Otsu-based initialization algorithm to optimize active contour models for skin lesion segmentation

BACKGROUND

Medical image processing has gained much attention in developing computer-aided diagnosis (CAD) of diseases. CAD systems require deep understanding of X-rays, MRIs, CT scans and other medical images. The segmentation of the region of interest (ROI) from those images is one of the most crucial tasks.

OBJECTIVE

Although active contour model (ACM) is a popular method to segment ROIs in medical images, the final segmentation results highly depend on the initial placement of the contour. In order to overcome this challenge, the objective of this study is to investigate feasibility of developing a fully automated initialization process that can be optimally used in ACM to more effectively segment ROIs.

METHODS

In this study, a fully automated initialization algorithm namely, an adaptive Otsu-based initialization (AOI) method is proposed. Using this proposed method, an initial contour is produced and further refined by the ACM to produce accurate segmentation. For evaluation of the proposed algorithm, the ISIC-2017 Skin Lesion dataset is used due to its challenging complexities.

RESULTS

Four different supervised performance evaluation metrics are employed to measure the accuracy and robustness of the proposed algorithm. Using this AOI algorithm, the ACM significantly (p≤0.05) outperforms Otsu thresholding method with 0.88 Dice Score Coefficients (DSC) and 0.79 Jaccard Index (JI) and computational complexity of 0(mn).

CONCLUSIONS

After comparing proposed method with other state-of-the-art methods, our study demonstrates that the proposed methods is superior to other skin lesion segmentation methods, and it requires no training time, which also makes the new method more efficient than other deep learning and machine learning methods.

Metaheuristic based Optimization Methods for the Segmentation of Tuberculosis Sputum Smear Images

Tuberculosis (TB) is a worldwide health crisis and second primary infectious disease that causes death. An attempt has been made to detect the presence of bacilli in sputum smears. The smear images recorded under standard image acquisition protocol are segmented by metaheuristic-based methods. Morphological operators are embedded in Ant Colony Optimization (ACO) and Particle Swarm Optimization (PSO) segmentation to retain concavity of rod-shaped bacilli. Results demonstrate that hybrid ACO segmentation is able to retain the shape of bacilli in TB images. Segmented images are validated with ground truth using overlap, distance and probability-based measures. Significant shape-based features such as area, perimeter, compactness, shape factor and tortuosity are extracted from the segmented images. It is observed that hybrid method preserves more edges, detects the presence of bacilli and facilitates direct segmentation with reduced number of redundant searches to generate edges. Thus this hybrid ACO-morphology segmentation technique aid in the diagnostic relevance of TB images.

Unified Focal loss: Generalising Dice and cross entropy-based losses to handle class imbalanced medical image segmentation

Automatic segmentation methods are an important advancement in medical image analysis. Machine learning techniques, and deep neural networks in particular, are the state-of-the-art for most medical image segmentation tasks. Issues with class imbalance pose a significant challenge in medical datasets, with lesions often occupying a considerably smaller volume relative to the background. Loss functions used in the training of deep learning algorithms differ in their robustness to class imbalance, with direct consequences for model convergence. The most commonly used loss functions for segmentation are based on either the cross entropy loss, Dice loss or a combination of the two. We propose the Unified Focal loss, a new hierarchical framework that generalises Dice and cross entropy-based losses for handling class imbalance. We evaluate our proposed loss function on five publicly available, class imbalanced medical imaging datasets: CVC-ClinicDB, Digital Retinal Images for Vessel Extraction (DRIVE), Breast Ultrasound 2017 (BUS2017), Brain Tumour Segmentation 2020 (BraTS20) and Kidney Tumour Segmentation 2019 (KiTS19). We compare our loss function performance against six Dice or cross entropy-based loss functions, across 2D binary, 3D binary and 3D multiclass segmentation tasks, demonstrating that our proposed loss function is robust to class imbalance and consistently outperforms the other loss functions. Source code is available at: https://github.com/mlyg/unified-focal-loss.

•

Loss function choice is crucial for class-imbalanced medical imaging datasets.

•

Understanding the relationship between loss functions is key to inform choice.

•

Unified Focal loss generalises Dice and cross-entropy based loss functions.

•

Unified Focal loss outperforms various Dice and cross-entropy based loss functions.

Domain-Adaptive Artificial Intelligence-Based Model for Personalized Diagnosis of Trivial Lesions Related to COVID-19 in Chest Computed Tomography Scans

BACKGROUND

Early and accurate detection of COVID-19-related findings (such as well-aerated regions, ground-glass opacity, crazy paving and linear opacities, and consolidation in lung computed tomography (CT) scan) is crucial for preventive measures and treatment. However, the visual assessment of lung CT scans is a time-consuming process particularly in case of trivial lesions and requires medical specialists.

METHOD

A recent breakthrough in deep learning methods has boosted the diagnostic capability of computer-aided diagnosis (CAD) systems and further aided health professionals in making effective diagnostic decisions. In this study, we propose a domain-adaptive CAD framework, namely the dilated aggregation-based lightweight network (DAL-Net), for effective recognition of trivial COVID-19 lesions in CT scans. Our network design achieves a fast execution speed (inference time is 43 ms on a single image) with optimal memory consumption (almost 9 MB). To evaluate the performances of the proposed and state-of-the-art models, we considered two publicly accessible datasets, namely COVID-19-CT-Seg (comprising a total of 3520 images of 20 different patients) and MosMed (including a total of 2049 images of 50 different patients).

RESULTS

Our method exhibits average area under the curve (AUC) up to 98.84%, 98.47%, and 95.51% for COVID-19-CT-Seg, MosMed, and cross-dataset, respectively, and outperforms various state-of-the-art methods.

CONCLUSIONS

These results demonstrate that deep learning-based models are an effective tool for building a robust CAD solution based on CT data in response to present disaster of COVID-19.

Magnetic Resonance Imaging Segmentation via Weighted Level Set Model Based on Local Kernel Metric and Spatial Constraint

Magnetic resonance imaging (MRI) segmentation is a fundamental and significant task since it can guide subsequent clinic diagnosis and treatment. However, images are often corrupted by defects such as low-contrast, noise, intensity inhomogeneity, and so on. Therefore, a weighted level set model (WLSM) is proposed in this study to segment inhomogeneous intensity MRI destroyed by noise and weak boundaries. First, in order to segment the intertwined regions of brain tissue accurately, a weighted neighborhood information measure scheme based on local multi information and kernel function is designed. Then, the membership function of fuzzy c-means clustering is used as the spatial constraint of level set model to overcome the sensitivity of level set to initialization, and the evolution of level set function can be adaptively changed according to different tissue information. Finally, the distance regularization term in level set function is replaced by a double potential function to ensure the stability of the energy function in the evolution process. Both real and synthetic MRI images can show the effectiveness and performance of WLSM. In addition, compared with several state-of-the-art models, segmentation accuracy and Jaccard similarity coefficient obtained by WLSM are increased by 0.0586, 0.0362 and 0.1087, 0.0703, respectively.