1
|
Yao B, Tiwari P, Li Q. Self-supervised pre-trained neural network for quantum natural language processing. Neural Netw 2025; 184:107004. [PMID: 39671984 DOI: 10.1016/j.neunet.2024.107004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2024] [Revised: 11/02/2024] [Accepted: 12/01/2024] [Indexed: 12/15/2024]
Abstract
Quantum computing models have propelled advances in many application domains. However, in the field of natural language processing (NLP), quantum computing models are limited in representation capacity due to the high linearity of the underlying quantum computing architecture. This work attempts to address this limitation by leveraging the concept of self-supervised pre-training, a paradigm that has been propelling the rocketing development of NLP, to increase the power of quantum NLP models on the representation level. Specifically, we present a self-supervised pre-training approach to train quantum encodings of sentences, and fine-tune quantum circuits for downstream tasks on its basis. Experiments show that pre-trained mechanism brings remarkable improvement over end-to-end pure quantum models, yielding meaningful prediction results on a variety of downstream text classification datasets.
Collapse
Affiliation(s)
- Ben Yao
- Department of Computer Science, University of Copenhagen, Copenhagen, Denmark.
| | - Prayag Tiwari
- School of Information Technology, Halmstad University, Halmstad, Sweden.
| | - Qiuchi Li
- Department of Computer Science, University of Copenhagen, Copenhagen, Denmark.
| |
Collapse
|
2
|
Fan F, Shi Y, Guggemos T, Zhu XX. Hybrid Quantum-Classical Convolutional Neural Network Model for Image Classification. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:18145-18159. [PMID: 37721886 DOI: 10.1109/tnnls.2023.3312170] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/20/2023]
Abstract
Image classification plays an important role in remote sensing. Earth observation (EO) has inevitably arrived in the big data era, but the high requirement on computation power has already become a bottleneck for analyzing large amounts of remote sensing data with sophisticated machine learning models. Exploiting quantum computing might contribute to a solution to tackle this challenge by leveraging quantum properties. This article introduces a hybrid quantum-classical convolutional neural network (QC-CNN) that applies quantum computing to effectively extract high-level critical features from EO data for classification purposes. Besides that, the adoption of the amplitude encoding technique reduces the required quantum bit resources. The complexity analysis indicates that the proposed model can accelerate the convolutional operation in comparison with its classical counterpart. The model's performance is evaluated with different EO benchmarks, including Overhead-MNIST, So2Sat LCZ42, PatternNet, RSI-CB256, and NaSC-TG2, through the TensorFlow Quantum platform, and it can achieve better performance than its classical counterpart and have higher generalizability, which verifies the validity of the QC-CNN model on EO data classification tasks.
Collapse
|
3
|
Wang Z, Yang L, Sun T, Yan W. Fusion PCAM R-CNN of Automatic Segmentation for Magnetic Flux Leakage Defects. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:11424-11435. [PMID: 37027265 DOI: 10.1109/tnnls.2023.3261363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
Magnetic leakage detection technology plays an important role in the long-oil pipeline. Automatic segmentation of defecting images is crucial for the detection of magnetic flux leakage (MFL) works. At present, accurate segmentation for small defects has always been a difficult problem. In contrast to the state-of-the-art MFL detection methodologies based on convolution neural network (CNN), an optimization method is devised in our study by integrating mask region-based CNN (Mask R-CNN) and information entropy constraint (IEC). To be precise, the principal component analysis (PCA) is utilized to improve the feature learning and network segmentation ability of the convolution kernel. The similarity constraint rule of information entropy is proposed to be inserted into the convolution layer in the Mask R-CNN network. The Mask R-CNN optimizes the convolutional kernel with similar weights or higher similarity, meanwhile, the PCA network reduces the dimension of the feature image to reconstruct the original feature vector. As such, the feature extraction of MFL defects is optimized in the convolution check. The research results can be applied in the field of MFL detection.
Collapse
|
4
|
Konar D, Bhattacharyya S, Gandhi TK, Panigrahi BK, Jiang R. 3-D Quantum-Inspired Self-Supervised Tensor Network for Volumetric Segmentation of Medical Images. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:10312-10325. [PMID: 37022399 DOI: 10.1109/tnnls.2023.3240238] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/19/2023]
Abstract
This article introduces a novel shallow 3-D self-supervised tensor neural network in quantum formalism for volumetric segmentation of medical images with merits of obviating training and supervision. The proposed network is referred to as the 3-D quantum-inspired self-supervised tensor neural network (3-D-QNet). The underlying architecture of 3-D-QNet is composed of a trinity of volumetric layers, viz., input, intermediate, and output layers interconnected using an S -connected third-order neighborhood-based topology for voxelwise processing of 3-D medical image data, suitable for semantic segmentation. Each of the volumetric layers contains quantum neurons designated by qubits or quantum bits. The incorporation of tensor decomposition in quantum formalism leads to faster convergence of network operations to preclude the inherent slow convergence problems faced by the classical supervised and self-supervised networks. The segmented volumes are obtained once the network converges. The suggested 3-D-QNet is tailored and tested on the BRATS 2019 Brain MR image dataset and the Liver Tumor Segmentation Challenge (LiTS17) dataset extensively in our experiments. The 3-D-QNet has achieved promising dice similarity (DS) as compared with the time-intensive supervised convolutional neural network (CNN)-based models, such as 3-D-UNet, voxelwise residual network (VoxResNet), Dense-Res-Inception Net (DRINet), and 3-D-ESPNet, thereby showing a potential advantage of our self-supervised shallow network on facilitating semantic segmentation.
Collapse
|
5
|
Qian Y, Wang X, Du Y, Wu X, Tao D. The Dilemma of Quantum Neural Networks. IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS 2024; 35:5603-5615. [PMID: 36191113 DOI: 10.1109/tnnls.2022.3208313] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/16/2023]
Abstract
The core of quantum machine learning is to devise quantum models with good trainability and low generalization error bounds than their classical counterparts to ensure better reliability and interpretability. Recent studies confirmed that quantum neural networks (QNNs) have the ability to achieve this goal on specific datasets. In this regard, it is of great importance to understand whether these advantages are still preserved on real-world tasks. Through systematic numerical experiments, we empirically observe that current QNNs fail to provide any benefit over classical learning models. Concretely, our results deliver two key messages. First, QNNs suffer from the severely limited effective model capacity, which incurs poor generalization on real-world datasets. Second, the trainability of QNNs is insensitive to regularization techniques, which sharply contrasts with the classical scenario. These empirical results force us to rethink the role of current QNNs and to design novel protocols for solving real-world problems with quantum advantages.
Collapse
|
6
|
Mostafa AM, Zakariah M, Aldakheel EA. Brain Tumor Segmentation Using Deep Learning on MRI Images. Diagnostics (Basel) 2023; 13:diagnostics13091562. [PMID: 37174953 PMCID: PMC10177460 DOI: 10.3390/diagnostics13091562] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Revised: 04/18/2023] [Accepted: 04/21/2023] [Indexed: 05/15/2023] Open
Abstract
Brain tumor (BT) diagnosis is a lengthy process, and great skill and expertise are required from radiologists. As the number of patients has expanded, so has the amount of data to be processed, making previous techniques both costly and ineffective. Many academics have examined a range of reliable and quick techniques for identifying and categorizing BTs. Recently, deep learning (DL) methods have gained popularity for creating computer algorithms that can quickly and reliably diagnose or segment BTs. To identify BTs in medical images, DL permits a pre-trained convolutional neural network (CNN) model. The suggested magnetic resonance imaging (MRI) images of BTs are included in the BT segmentation dataset, which was created as a benchmark for developing and evaluating algorithms for BT segmentation and diagnosis. There are 335 annotated MRI images in the collection. For the purpose of developing and testing BT segmentation and diagnosis algorithms, the brain tumor segmentation (BraTS) dataset was produced. A deep CNN was also utilized in the model-building process for segmenting BTs using the BraTS dataset. To train the model, a categorical cross-entropy loss function and an optimizer, such as Adam, were employed. Finally, the model's output successfully identified and segmented BTs in the dataset, attaining a validation accuracy of 98%.
Collapse
Affiliation(s)
- Almetwally M Mostafa
- Department of Information Systems, College of Computer and Information Sciences, King Saud University, P.O. Box 51178, Riyadh 11543, Saudi Arabia
| | - Mohammed Zakariah
- Department of Computer Science, College of Computer and Information Science, King Saud University, P.O. Box 51178, Riyadh 11543, Saudi Arabia
| | - Eman Abdullah Aldakheel
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| |
Collapse
|
7
|
ZainEldin H, Gamel SA, El-Kenawy ESM, Alharbi AH, Khafaga DS, Ibrahim A, Talaat FM. Brain Tumor Detection and Classification Using Deep Learning and Sine-Cosine Fitness Grey Wolf Optimization. BIOENGINEERING (BASEL, SWITZERLAND) 2022; 10:bioengineering10010018. [PMID: 36671591 PMCID: PMC9854739 DOI: 10.3390/bioengineering10010018] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/17/2022] [Revised: 11/12/2022] [Accepted: 12/17/2022] [Indexed: 12/24/2022]
Abstract
Diagnosing a brain tumor takes a long time and relies heavily on the radiologist's abilities and experience. The amount of data that must be handled has increased dramatically as the number of patients has increased, making old procedures both costly and ineffective. Many researchers investigated a variety of algorithms for detecting and classifying brain tumors that were both accurate and fast. Deep Learning (DL) approaches have recently been popular in developing automated systems capable of accurately diagnosing or segmenting brain tumors in less time. DL enables a pre-trained Convolutional Neural Network (CNN) model for medical images, specifically for classifying brain cancers. The proposed Brain Tumor Classification Model based on CNN (BCM-CNN) is a CNN hyperparameters optimization using an adaptive dynamic sine-cosine fitness grey wolf optimizer (ADSCFGWO) algorithm. There is an optimization of hyperparameters followed by a training model built with Inception-ResnetV2. The model employs commonly used pre-trained models (Inception-ResnetV2) to improve brain tumor diagnosis, and its output is a binary 0 or 1 (0: Normal, 1: Tumor). There are primarily two types of hyperparameters: (i) hyperparameters that determine the underlying network structure; (ii) a hyperparameter that is responsible for training the network. The ADSCFGWO algorithm draws from both the sine cosine and grey wolf algorithms in an adaptable framework that uses both algorithms' strengths. The experimental results show that the BCM-CNN as a classifier achieved the best results due to the enhancement of the CNN's performance by the CNN optimization's hyperparameters. The BCM-CNN has achieved 99.98% accuracy with the BRaTS 2021 Task 1 dataset.
Collapse
Affiliation(s)
- Hanaa ZainEldin
- Computer Engineering and Control Systems Department, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt
| | - Samah A. Gamel
- Computer Engineering and Control Systems Department, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt
| | - El-Sayed M. El-Kenawy
- Department of Communications and Electronics, Delta Higher Institute of Engineering and Technology, Mansoura 35111, Egypt
- Correspondence: (E.-S.M.E.-K.); (D.S.K.); (A.I.)
| | - Amal H. Alharbi
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
| | - Doaa Sami Khafaga
- Department of Computer Sciences, College of Computer and Information Sciences, Princess Nourah bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia
- Correspondence: (E.-S.M.E.-K.); (D.S.K.); (A.I.)
| | - Abdelhameed Ibrahim
- Computer Engineering and Control Systems Department, Faculty of Engineering, Mansoura University, Mansoura 35516, Egypt
- Correspondence: (E.-S.M.E.-K.); (D.S.K.); (A.I.)
| | - Fatma M. Talaat
- Machine Learning & Information Retrieval Department, Faculty of Artificial Intelligence, Kafrelsheikh University, Kafrelsheikh 33511, Egypt
| |
Collapse
|
8
|
A secure two-qubit quantum model for segmentation and classification of brain tumor using MRI images based on blockchain. Neural Comput Appl 2022. [DOI: 10.1007/s00521-022-07388-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
|
9
|
Konar D, Bhattacharyya S, Dey S, Panigrahi BK. Optimized activation for quantum-inspired self-supervised neural network based fully automated brain lesion segmentation. APPL INTELL 2022. [DOI: 10.1007/s10489-021-03108-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
|
10
|
Abstract
AbstractBrain tumor occurs owing to uncontrolled and rapid growth of cells. If not treated at an initial phase, it may lead to death. Despite many significant efforts and promising outcomes in this domain, accurate segmentation and classification remain a challenging task. A major challenge for brain tumor detection arises from the variations in tumor location, shape, and size. The objective of this survey is to deliver a comprehensive literature on brain tumor detection through magnetic resonance imaging to help the researchers. This survey covered the anatomy of brain tumors, publicly available datasets, enhancement techniques, segmentation, feature extraction, classification, and deep learning, transfer learning and quantum machine learning for brain tumors analysis. Finally, this survey provides all important literature for the detection of brain tumors with their advantages, limitations, developments, and future trends.
Collapse
|