Medical visual question answering via corresponding feature fusion combined with semantic attention

Weakly supervised salient object detection via image category annotation

The rapid development of deep learning has made a great progress in salient object detection task. Fully supervised methods need a large number of pixel-level annotations. To avoid laborious and consuming annotation, weakly supervised methods consider low-cost annotations such as category, bounding-box, scribble, etc. Due to simple annotation and existing large-scale classification datasets, the category annotation based methods have received more attention while still suffering from inaccurate detection. In this work, we proposed one weakly supervised method with category annotation. First, we proposed one coarse object location network (COLN) to roughly locate the object of an image with category annotation. Second, we refined the coarse object location to generate pixel-level pseudo-labels and proposed one quality check strategy to select high quality pseudo labels. To this end, we studied COLN twice followed by refinement to obtain a pseudo-labels pair and calculated the consistency of pseudo-label pairs to select high quality labels. Third, we proposed one multi-decoder neural network (MDN) for saliency detection supervised by pseudo-label pairs. The loss of each decoder and between decoders are both considered. Last but not least, we proposed one pseudo-labels update strategy to iteratively optimize pseudo-labels and saliency detection models. Performance evaluation on four public datasets shows that our method outperforms other image category annotation based work.

Dual-process system based on mixed semantic fusion for Chinese medical knowledge-based question answering

Chinese medical knowledge-based question answering (cMed-KBQA) is a vital component of the intelligence question-answering assignment. Its purpose is to enable the model to comprehend questions and then deduce the proper answer from the knowledge base. Previous methods solely considered how questions and knowledge base paths were represented, disregarding their significance. Due to entity and path sparsity, the performance of question and answer cannot be effectively enhanced. To address this challenge, this paper presents a structured methodology for the cMed-KBQA based on the cognitive science dual systems theory by synchronizing an observation stage (System 1) and an expressive reasoning stage (System 2). System 1 learns the question's representation and queries the associated simple path. Then System 2 retrieves complicated paths for the question from the knowledge base by using the simple path provided by System 1. Specifically, System 1 is implemented by the entity extraction module, entity linking module, simple path retrieval module, and simple path-matching model. Meanwhile, System 2 is performed by using the complex path retrieval module and complex path-matching model. The public CKBQA2019 and CKBQA2020 datasets were extensively studied to evaluate the suggested technique. Using the metric average F1-score, our model achieved 78.12% on CKBQA2019 and 86.60% on CKBQA2020.

A comprehensive interpretation for medical VQA: Datasets, techniques, and challenges

Recent technological developments and improvement in the medical domain demands advancement, to address the issue of early disease detection. Also, the current pandemic has resulted in considerable progress of improvement in the medical domain, through online consultation by physicians for different diseases using clinical reports and medical images. A similar process is adopted in developing a Visual Question Answering (VQA) system in the medical field. In this paper, existing medical VQA datasets, appropriate techniques, suitable quantitative metrics, real time challenges and, the implementation of one VQA approach with algorithms and performance evaluation are discussed. The medical VQA datasets collected from multiple sources are represented in different perspectives (organwise, planewise, modality-type and abnormality-type) for a better understanding and visualization. Then the techniques used in VQA are subsequently grouped and explained, based on evolution, complexity in the dataset and the need for semantics in understanding the questions. In addition, the implementation of a VQA approach using VGGNet and LSTM is carried out for existing and improved datasets, and analyzed with accuracy and BLEU score metrics. The improved datasets, created through dataset reduction and augmentation approaches, resulted in better performance than the existing datasets. Finally, the challenges of the medical VQA domain are examined in terms of datasets, combining techniques, and modifying the parameters of existing performance metrics for future research.