Weakly supervised image segmentation beyond tight bounding box annotations

Weakly supervised image segmentation approaches in the literature usually achieve high segmentation performance using tight bounding box supervision and decrease the performance greatly when supervised by loose bounding boxes. However, compared with loose bounding box, it is much more difficult to acquire tight bounding box due to its strict requirements on the precise locations of the four sides of the box. To resolve this issue, this study investigates whether it is possible to maintain good segmentation performance when loose bounding boxes are used as supervision. For this purpose, this work extends our previous parallel transformation based multiple instance learning (MIL) for tight bounding box supervision by integrating an MIL strategy based on polar transformation to assist image segmentation. The proposed polar transformation based MIL formulation works for both tight and loose bounding boxes, in which a positive bag is defined as pixels in a polar line of a bounding box with one endpoint located inside the object enclosed by the box and the other endpoint located at one of the four sides of the box. Moreover, a weighted smooth maximum approximation is introduced to incorporate the observation that pixels closer to the origin of the polar transformation are more likely to belong to the object in the box. The proposed approach was evaluated on two public datasets using dice coefficient when bounding boxes at different precision levels were considered in the experiments. The results demonstrate that the proposed approach achieves state-of-the-art performance for bounding boxes at all precision levels and is robust to mild and moderate errors in the loose bounding box annotations. The codes are available at https://github.com/wangjuan313/wsis-beyond-tightBB.

An automatic system for extracting figure-caption pair from medical documents: a six-fold approach

Background

Figures and captions in medical documentation contain important information. As a result, researchers are becoming more interested in obtaining published medical figures from medical papers and utilizing the captions as a knowledge source.

Methods

This work introduces a unique and successful six-fold methodology for extracting figure-caption pairs. The A-torus wavelet transform is used to retrieve the first edge from the scanned page. Then, using the maximally stable extremal regions connected component feature, text and graphical contents are isolated from the edge document, and multi-layer perceptron is used to successfully detect and retrieve figures and captions from medical records. The figure-caption pair is then extracted using the bounding box approach. The files that contain the figures and captions are saved separately and supplied to the end useras theoutput of any investigation. The proposed approach is evaluated using a self-created database based on the pages collected from five open access books: Sergey Makarov, Gregory Noetscher and Aapo Nummenmaa's book "Brain and Human Body Modelling 2021", "Healthcare and Disease Burden in Africa" by Ilha Niohuru, "All-Optical Methods to Study Neuronal Function" by Eirini Papagiakoumou, "RNA, the Epicenter of Genetic Information" by John Mattick and Paulo Amaral and "Illustrated Manual of Pediatric Dermatology" by Susan Bayliss Mallory, Alanna Bree and Peggy Chern.

Results

Experiments and findings comparing the new method to earlier systems reveal a significant increase in efficiency, demonstrating the suggested technique's robustness and efficiency.

An Automated Segmentation of Leukocytes Using Modified Watershed Algorithm on Peripheral Blood Smear Images

Leukemia can be detected by an abnormal rise in the number of immature lymphocytes and by a decrease in the number of other blood cells. To diagnose leukemia, image processing techniques are utilized to examine microscopic peripheral blood smear (PBS) images automatically and swiftly. To the best of our knowledge, the initial step in subsequent processing is a robust segmentation technique for identifying leukocytes from their surroundings. The paper presents the segmentation of leukocytes in which three color spaces are considered in this study for image enhancement. The proposed algorithm uses a marker-based watershed algorithm and peak local maxima. The algorithm was used on three different datasets with various color tones, image resolutions, and magnifications. The average precision for all three-color spaces was the same, i.e. 94% but the Structural Similarity Index Metric (SSIM) and recall of HSV were better than other two. The results of this study will aid experts in narrowing down their options for segmenting leukemia. Based on the comparison, it was concluded that when the colour space correction technique is used, the accuracy of the proposed methodology improves.

BBox-Guided Segmentor: Leveraging expert knowledge for accurate stroke lesion segmentation using weakly supervised bounding box prior

Stroke is one of the leading causes of death and disability in the world. Despite intensive research on automatic stroke lesion segmentation from non-invasive imaging modalities including diffusion-weighted imaging (DWI), challenges remain such as a lack of sufficient labeled data for training deep learning models and failure in detecting small lesions. In this paper, we propose BBox-Guided Segmentor, a method that significantly improves the accuracy of stroke lesion segmentation by leveraging expert knowledge. Specifically, our model uses a very coarse bounding box label provided by the expert and then performs accurate segmentation automatically. The small overhead of having the expert provide a rough bounding box leads to large performance improvement in segmentation, which is paramount to accurate stroke diagnosis. To train our model, we employ a weakly-supervised approach that uses a large number of weakly-labeled images with only bounding boxes and a small number of fully labeled images. The scarce fully labeled images are used to train a generator segmentation network, while adversarial training is used to leverage the large number of weakly-labeled images to provide additional learning signals. We evaluate our method extensively using a unique clinical dataset of 99 fully labeled cases (i.e., with full segmentation map labels) and 831 weakly labeled cases (i.e., with only bounding box labels), and the results demonstrate the superior performance of our approach over state-of-the-art stroke lesion segmentation models. We also achieve competitive performance as a SOTA fully supervised method using less than one-tenth of the complete labels. Our proposed approach has the potential to improve stroke diagnosis and treatment planning, which may lead to better patient outcomes.

Measurement of Endotracheal Tube Positioning on Chest X-Ray Using Object Detection

Patients who are intubated with endotracheal tubes often receive chest x-ray (CXR) imaging to determine whether the tube is correctly positioned. When these CXRs are interpreted by a radiologist, they evaluate whether the tube needs to be repositioned and typically provide a measurement in centimeters between the endotracheal tube tip and carina. In this project, a large dataset of endotracheal tube and carina bounding boxes was annotated on CXRs, and a machine-learning model was trained to generate these boxes on new CXRs and to calculate a distance measurement between the tube and carina. This model was applied to a gold standard annotated dataset, as well as to all prospective data passing through our radiology system for two weeks. Inter-radiologist variability was also measured on a test dataset. The distance measurements for both the gold standard dataset (mean error = 0.70 cm) and prospective dataset (mean error = 0.68 cm) were noninferior to inter-radiologist variability (mean error = 0.70 cm) within an equivalence bound of 0.1 cm. This suggests that this model performs at an accuracy similar to human measurements, and these distance calculations can be used for clinical report auto-population and/or worklist prioritization of severely malpositioned tubes.

Weakly-supervised convolutional neural networks of renal tumor segmentation in abdominal CTA images

Background

Renal cancer is one of the 10 most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) is an effective way to treat renal cancer. Localization and delineation of the renal tumor from pre-operative CT Angiography (CTA) is an important step for LPN surgery planning. Recently, with the development of the technique of deep learning, deep neural networks can be trained to provide accurate pixel-wise renal tumor segmentation in CTA images. However, constructing the training dataset with a large amount of pixel-wise annotations is a time-consuming task for the radiologists. Therefore, weakly-supervised approaches attract more interest in research.

Methods

In this paper, we proposed a novel weakly-supervised convolutional neural network (CNN) for renal tumor segmentation. A three-stage framework was introduced to train the CNN with the weak annotations of renal tumors, i.e. the bounding boxes of renal tumors. The framework includes pseudo masks generation, group and weighted training phases. Clinical abdominal CT angiographic images of 200 patients were applied to perform the evaluation.

Results

Extensive experimental results show that the proposed method achieves a higher dice coefficient (DSC) of 0.826 than the other two existing weakly-supervised deep neural networks. Furthermore, the segmentation performance is close to the fully supervised deep CNN.

Conclusions

The proposed strategy improves not only the efficiency of network training but also the precision of the segmentation.

Localization of common carotid artery transverse section in B-mode ultrasound images using faster RCNN: a deep learning approach

Cardiologists can acquire important information related to patients' cardiac health using carotid artery stiffness, its lumen diameter (LD), and its carotid intima-media thickness (cIMT). The sonographers primarily concern about the location of the artery in B-mode ultrasound images. Localization using manual methods is tedious and time-consuming and also may lead to some errors. On the other hand, automated approaches are more objective and can provide the localization of the artery at near real time. Above arterial parameters may be determined after localization of the artery in real time.A novel method of localization of common carotid artery (CCA) transverse section is presented in this work. The method is known as fast region convolutional neural network (FRCNN)-based localization method and is designed using a stack of three layers viz. convolutional layers, fully connected layers, and pooling layers. These organized layers constitute a region proposal network (RPN) and an object class detection network (OCDN). We obtain an outcome as a bounding box along with a score of prediction around the cross-section of the CCA.B-mode ultrasound image database of CCA is split into training and testing set, to accomplish this, three partition methods K = 2, 5, and 10 are used in our work. The training is extended for 30, 200, and 2000 epochs in order to achieve fine-tuned features from the convolutional neural network. After 2000 epochs, we obtain 95% validation accuracy; however, mean of the accuracies up to 2000 epochs is 89.36% for K = 10 partitions protocol (training 90%, testing 10%). Generated CNN model is tested on a different dataset of 433 images and the acquired accuracy is 87.99%. Thus, the proposed method including an advanced deep learning technique demonstrates promising localization for carotid artery transverse section. Graphical abstract.