Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis

Global, regional, and national burdens of hip fractures in elderly individuals from 1990 to 2021 and predictions up to 2050: A systematic analysis of the Global Burden of Disease Study 2021

PURPOSE

We aimed to analyse the global, regional, and national burdens of hip fractures in older adults from 1990 to 2021, with projections to 2050, on the basis of data from the GBD 2021 study.

METHODS

We employed a joinpoint model to analyse trends in the burden of hip fractures from 1990‒2021. The estimated annual percentage change (EAPC) was used to quantify temporal trends over this period. We evaluated the relationship between the social development index and the burden of hip fracture in elderly people and conducted a health inequality analysis. Additionally, we applied Long-short Term Memory (LSTM) networks to forecast burden trends of hip fractures up to 2050.

RESULTS

The global age-standardized incidence rate (ASIR) for hip fractures in older adults rose from 781.56 per 100,000 in 1990 to 948.81 in 2021. The 2021 age-standardized prevalence rate (ASPR) was 1,894.07, and the age-standardized YLD rate (ASDR) was 173.52. From 1990 to 2021, the incidence and prevalence increased by 168.71 % and 173.07 %, respectively, while the burden of DALYs decreased. Future trends were projected via the LSTM. The burden and risk factors for hip fractures varied significantly by sex, country, and region. Population and aging are primary contributors to the rising incidence of elderly hip fractures, with falls being the leading direct cause.

CONCLUSION

From 1990 to 2021, the global burden of hip fractures in the elderly population, especially among older women, steadily increased. Population ageing highlights the urgent need for targeted public health interventions and resource allocation, including early diagnosis, effective prevention strategies, and region-specific management approaches.

Heuristic multi-scale feature fusion with attention-based CNN for sentiment analysis

The sentiment analysis is an essential component that enables automation of achieving insights from the information that is user generated. However, the difficulty of sentiment analysis is the lack of enough labelled data in the Natural Language Processing (NLP) sector. Thus, to evaluate these sentiments, multiple mechanisms have been utilized in the past decades. The deep learning-aided approaches are becoming very famous nowadays because of their better performances. To surmount such existing issues, an attention deep learning model is proposed using an improved heuristic approach. At first, the input text data is gathered from public resources. Further, it is followed by text pre-processing to prevent unrelated text data. Further, the obtained pre-processed text is fed into the Multiscale Feature Fusion-based Adaptive and Attention-based Convolution Neural Network (MFF-AACNet). In the developed system, the features are extracted from Bidirectional Encoder Representations from Transformers (BERT), Transformers, and word2vector. Furthermore, the resultant features are fused, and it is subjected to the MFF-AACNet, where the sentiment is analysed. The parameter tuning is done by an improved Fitness Opposition of Rat Swarm Optimizer (FORSO). Finally, the performance analysis was conducted for the implemented model. The proposed framework achieves higher accuracy compared to traditional methods.

Cross-Domain Social Rumor-Propagation Model Based on Transfer Learning

Rumors in different topic domains have different text characteristics but similar emotional tendencies. To resolve the scarce-data problem in some rumor-topic domains, this study proposes a cross-domain rumor-propagation model, which is based on transfer learning. First, given the diversity and complexity of the rumor-propagation landscape, this study introduces a novel method, User-Retweet-Rumor2vec (URR2vec), which leverages the power of representation learning to uncover latent features within rumor topics. It also displays the forwarding relationship between users and rumors, user node information, and rumor-topic information in low-dimensional space. To capture the impact of human emotional cognition during rumor spreading, we also introduce a deep-learning model based on the natural language texts of rumor topics, which analyzes the sentiment in the text and uncovers the emotional correlations among users. Furthermore, a rumor-propagation prediction model based on the text-sentiment analysis-graph convolutional network (TSA-GCN) is proposed and pre-trained on existing rumor-topic data to ensure its prediction accuracy. Finally, considering the data sparsity at a rumor-topic outbreak, the trained propagation model is transferred to the rumor topic for prediction. Meanwhile, the rumor topic in different domains has different edges and conditional distribution, similar emotional characteristics, and network structure among the rumor topics. After fine-tuning the parameter and adding a domain adaptation layer in TSA-GCN, a domain adaptation model based on parameter and graph-structure migration is obtained.

MG-SIN: Multigraph Sparse Interaction Network for Multitask Stance Detection

Stance detection on social media aims to identify if an individual is in support of or against a specific target. Most existing stance detection approaches primarily rely on modeling the contextual semantic information in sentences and neglect to explore the pragmatics dependency information of words, thus degrading performance. Although several single-task learning methods have been proposed to capture richer semantic representation information, they still suffer from semantic sparsity problems caused by short texts on social media. This article proposes a novel multigraph sparse interaction network (MG-SIN) by using multitask learning (MTL) to identify the stances and classify the sentiment polarities of tweets simultaneously. Our basic idea is to explore the pragmatics dependency relationship between tasks at the word level by constructing two types of heterogeneous graphs, including task-specific and task-related graphs (tr-graphs), to boost the learning of task-specific representations. A graph-aware module is proposed to adaptively facilitate information sharing between tasks via a novel sparse interaction mechanism among heterogeneous graphs. Through experiments on two real-world datasets, compared with the state-of-the-art baselines, the extensive results exhibit that MG-SIN achieves competitive improvements of up to 2.1% and 2.42% for the stance detection task, and 5.26% and 3.93% for the sentiment analysis task, respectively.

Multi task opinion enhanced hybrid BERT model for mental health analysis

Understanding the nuanced emotions and points of view included in user-generated content remains challenging, even though text data analysis for mental health is a crucial instrument for assessing emotional well-being. Most current models neglect the significance of integrating viewpoints in comprehending mental health in favor of single-task learning. To offer a more thorough knowledge of mental health, in this study, we present an Opinion-Enhanced Hybrid BERT Model (Opinion-BERT), built to handle multi-task learning for simultaneous sentiment and status categorization. With the help of TextBlob and SciPy, we extracted opinions and dynamically constructed new opinion embeddings to complement the pre-trained BERT model. Using a hybrid architecture, these embeddings are integrated with the contextual embeddings of BERT, whereby the CNN and BiGRU layers collected local and sequential characteristics. This combination helps our model to identify and categorize user status and attitudes from the text more accurately, which leads to more accurate mental health assessments. When we compared the performance of Opinion-BERT to some baseline models, including BERT, RoBERTa, and DistilBERT, we found that it performed much better. Opinion-enhanced embeddings are crucial for improving performance, as demonstrated by our multi-task learning framework's 96.77% sentiment classification accuracy of 94.22% status classification accuracy. This work provides a more nuanced understanding of emotions and psychological states by demonstrating the potential of combining opinion and sentiment data for mental health analysis in a multi-task learning environment.

Online comments of tourist attractions combining artificial intelligence text mining model and attention mechanism

This paper intends to solve the limitations of the existing methods to deal with the comments of tourist attractions. With the technical support of Artificial Intelligence (AI), an online comment method of tourist attractions based on text mining model and attention mechanism is proposed. In the process of text mining, the attention mechanism is used to calculate the contribution of each topic to text representation on the topic layer of Latent Dirichlet Allocation (LDA). The Bidirectional Recurrent Neural Network (BiGRU) can effectively capture the temporal relationship and semantic dependence in the text through its powerful sequence modeling ability, thus achieving a more accurate classification of emotional tendencies. In order to verify the performance of the proposed ATT-LDA- Bigelow model, online comments about tourist attractions are collected from Ctrip.com, and users' emotional tendencies towards different scenic spots are analyzed. The results show that this model has the best emotion classification effect in online comments of scenic spots, with the accuracy and F1 value reaching 93.85% and 93.68% respectively, which is superior to other emotion classification models. The proposed method not only improves the accuracy of sentiment analysis, but also provides strong support for the optimization of tourism recommendation system and provides more comprehensive, objective and accurate tourism information for scenic spot managers and tourism enterprises. This achievement is expected to bring new enlightenment and breakthrough to the research and practice in related fields.

DSTCNet: Deep Spectro-Temporal-Channel Attention Network for Speech Emotion Recognition

Speech emotion recognition (SER) plays an important role in human-computer interaction, which can provide better interactivity to enhance user experiences. Existing approaches tend to directly apply deep learning networks to distinguish emotions. Among them, the convolutional neural network (CNN) is the most commonly used method to learn emotional representations from spectrograms. However, CNN does not explicitly model features' associations in the spectral-, temporal-, and channel-wise axes or their relative relevance, which will limit the representation learning. In this article, we propose a deep spectro-temporal-channel network (DSTCNet) to improve the representational ability for speech emotion. The proposed DSTCNet integrates several spectro-temporal-channel (STC) attention modules into a general CNN. Specifically, we propose the STC module that infers a 3-D attention map along the dimensions of time, frequency, and channel. The STC attention can focus more on the regions of crucial time frames, frequency ranges, and feature channels. Finally, experiments were conducted on the Berlin emotional database (EmoDB) and interactive emotional dyadic motion capture (IEMOCAP) databases. The results reveal that our DSTCNet can outperform the traditional CNN-based and several state-of-the-art methods.

A Novel Public Sentiment Analysis Method Based on an Isomerism Learning Model via Multiphase Processing

The dissemination of public opinion in the social media network is driven by public sentiment, which can be used to promote the effective resolution of social incidents. However, public sentiments for incidents are often affected by environmental factors such as geography, politics, and ideology, which increases the complexity of the sentiment acquisition task. Therefore, a hierarchical mechanism is designed to reduce complexity and utilize processing at multiple phases to improve practicality. Through serial processing between different phases, the task of public sentiment acquisition can be decomposed into two subtasks, which are the classification of report text to locate incidents and sentiment analysis of individuals' reviews. Performance has been improved through improvements to the model structure, such as embedding tables and gating mechanisms. That being said, the traditional centralized structure model is not only easy to form model silos in the process of performing tasks but also faces security risks. In this article, a novel distributed deep learning model called isomerism learning based on blockchain is proposed to address these challenges, the trusted collaboration between models can be realized through parallel training. In addition, for the problem of text heterogeneity, we also designed a method to measure the objectivity of events to dynamically assign the weights of models to improve aggregation efficiency. Extensive experiments demonstrate that the proposed method can effectively improve performance and outperform the state-of-the-art methods significantly.

Hybrid Network Using Dynamic Graph Convolution and Temporal Self-Attention for EEG-Based Emotion Recognition

The electroencephalogram (EEG) signal has become a highly effective decoding target for emotion recognition and has garnered significant attention from researchers. Its spatial topological and time-dependent characteristics make it crucial to explore both spatial information and temporal information for accurate emotion recognition. However, existing studies often focus on either spatial or temporal aspects of EEG signals, neglecting the joint consideration of both perspectives. To this end, this article proposes a hybrid network consisting of a dynamic graph convolution (DGC) module and temporal self-attention representation (TSAR) module, which concurrently incorporates the representative knowledge of spatial topology and temporal context into the EEG emotion recognition task. Specifically, the DGC module is designed to capture the spatial functional relationships within the brain by dynamically updating the adjacency matrix during the model training process. Simultaneously, the TSAR module is introduced to emphasize more valuable time segments and extract global temporal features from EEG signals. To fully exploit the interactivity between spatial and temporal information, the hierarchical cross-attention fusion (H-CAF) module is incorporated to fuse the complementary information from spatial and temporal features. Extensive experimental results on the DEAP, SEED, and SEED-IV datasets demonstrate that the proposed method outperforms other state-of-the-art methods.

LLM4THP: a computing tool to identify tumor homing peptides by molecular and sequence representation of large language model based on two-layer ensemble model strategy

Tumor homing peptides (THPs) have a distinctive capacity to specifically attach to tumor cells, providing a promising approach for targeted cancer treatment and detection. Although THPs have the potential for significant impact, their detection by conventional methods is both time-consuming and expensive. To tackle this issue, we provide LLM4THP, an innovative computational approach that utilizes large language models (LLMs) to quickly and effectively detect THPs. LLM4THP utilizes two protein LLMs, ESM2 and Prot_T5_XL_UniRef50, to encode peptide sequences. This allows for the capture of complex patterns and relationships within the peptide data. In addition, we utilize inherent sequence characteristics such as Amino Acid Composition (AAC), Pseudo Amino Acid Composition (PAAC), Amphiphilic Pseudo Amino Acid Composition (APAAC), and Composition, Transition, and Distribution (CTD) to improve the representation of peptides. The RDKitDescriptors feature representation approach transforms peptide sequences into molecular objects and computes chemical characteristics, resulting in enhanced THP identification. The LLM4THP ensemble strategy incorporates various features into a two-layer learning architecture. The first layer consists of LightGBM, XGBoost, Random Forest, and Extremely Randomized Trees, which generate a set of meta results. The second layer utilizes Logistic Regression to further refine the identification of sequences as either THP or non-THP. LLM4THP exhibits exceptional performance compared to the most advanced methods, showcasing enhancements in accuracy, Matthew's correlation coefficient, F1 score, area under the curve, and average precision. The source code and dataset can be accessed at the following URL: https://github.com/abcair/LLM4THP.

Description-Enhanced Label Embedding Contrastive Learning for Text Classification

Text classification is one of the fundamental tasks in natural language processing, which requires an agent to determine the most appropriate category for input sentences. Recently, deep neural networks have achieved impressive performance in this area, especially pretrained language models (PLMs). Usually, these methods concentrate on input sentences and corresponding semantic embedding generation. However, for another essential component: labels, most existing works either treat them as meaningless one-hot vectors or use vanilla embedding methods to learn label representations along with model training, underestimating the semantic information and guidance that these labels reveal. To alleviate this problem and better exploit label information, in this article, we employ self-supervised learning (SSL) in model learning process and design a novel self-supervised relation of relation ( [Formula: see text]) classification task for label utilization from a one-hot manner perspective. Then, we propose a novel relation of relation learning network( [Formula: see text]-Net) for text classification, in which text classification and [Formula: see text] classification are treated as optimization targets. Meanwhile, triplet loss is employed to enhance the analysis of differences and connections among labels. Moreover, considering that one-hot usage is still short of exploiting label information, we incorporate external knowledge from WordNet to obtain multiaspect descriptions for label semantic learning and extend [Formula: see text]-Net to a novel description-enhanced label embedding network(DELE) from a label embedding perspective. One step further, since these fine-grained descriptions may introduce unexpected noise, we develop a mutual interaction module to select appropriate parts from input sentences and labels simultaneously based on contrastive learning (CL) for noise mitigation. Extensive experiments on different text classification tasks reveal that [Formula: see text]-Net can effectively improve the classification performance and DELE can make better use of label information and further improve the performance. As a byproduct, we have released the codes to facilitate other research.

The Prediction and Evaluation of Surface Quality during the Milling of Blade-Root Grooves Based on a Long Short-Term Memory Network and Signal Fusion

The surface quality of milled blade-root grooves in industrial turbine blades significantly influences their mechanical properties. The surface texture reveals the interaction between the tool and the workpiece during the machining process, which plays a key role in determining the surface quality. In addition, there is a significant correlation between acoustic vibration signals and surface texture features. However, current research on surface quality is still relatively limited, and most considers only a single signal. In this paper, 160 sets of industrial field data were collected by multiple sensors to study the surface quality of a blade-root groove. A surface texture feature prediction method based on acoustic vibration signal fusion is proposed to evaluate the surface quality. Fast Fourier transform (FFT) is used to process the signal, and the clean and smooth features are extracted by combining wavelet denoising and multivariate smoothing denoising. At the same time, based on the gray-level co-occurrence matrix, the surface texture image features of different angles of the blade-root groove are extracted to describe the texture features. The fused acoustic vibration signal features are input, and the texture features are output to establish a texture feature prediction model. After predicting the texture features, the surface quality is evaluated by setting a threshold value. The threshold is selected based on all sample data, and the final judgment accuracy is 90%.

Residual Sketch Learning for a Feature-Importance-Based and Linguistically Interpretable Ensemble Classifier

Motivated by both the commonly used "from wholly coarse to locally fine" cognitive behavior and the recent finding that simple yet interpretable linear regression model should be a basic component of a classifier, a novel hybrid ensemble classifier called hybrid Takagi-Sugeno-Kang fuzzy classifier (H-TSK-FC) and its residual sketch learning (RSL) method are proposed. H-TSK-FC essentially shares the virtues of both deep and wide interpretable fuzzy classifiers and simultaneously has both feature-importance-based and linguistic-based interpretabilities. RSL method is featured as follows: 1) a global linear regression subclassifier on all original features of all training samples is generated quickly by the sparse representation-based linear regression subclassifier training procedure to identify/understand the importance of each feature and partition the output residuals of the incorrectly classified training samples into several residual sketches; 2) by using both the enhanced soft subspace clustering method (ESSC) for the linguistically interpretable antecedents of fuzzy rules and the least learning machine (LLM) for the consequents of fuzzy rules on residual sketches, several interpretable Takagi-Sugeno-Kang (TSK) fuzzy subclassifiers are stacked in parallel through residual sketches and accordingly generated to achieve local refinements; and 3) the final predictions are made to further enhance H-TSK-FC's generalization capability and decide which interpretable prediction route should be used by taking the minimal-distance-based priority for all the constructed subclassifiers. In contrast to existing deep or wide interpretable TSK fuzzy classifiers, benefiting from the use of feature-importance-based interpretability, H-TSK-FC has been experimentally witnessed to have faster running speed and better linguistic interpretability (i.e., fewer rules and/or TSK fuzzy subclassifiers and smaller model complexities) yet keep at least comparable generalization capability.

Deep parallel contextual analysis framework based emotion prediction in community wellness communications on social media

Understanding public emotion on social media about community wellness is crucial for enhancing health awareness and guiding policy-making. In order to more fully mine the deep contextual semantical information of short texts and further enhance the effectiveness of emotion prediction in social media, we propose the Deep Parallel Contextual Analysis Framework (DPCAF) in the community wellness domain, specifically addressing the challenges of limited text length and available semantical features in social media text. Specifically, at the embedding layer, we first utilize two different word embedding techniques to generate high-quality vector representations, aiming to achieve more comprehensive semantical capture, stronger generalization ability, and more robust model performance. Subsequently, in the deep contextual layer, the obtained representations are fused with POS and locational representations, and processed through a deep parallel layer composed of Convolutional Neural Networks and Bidirectional Long Short-Term Memory Network. An attention model is then used to further extract semantical features of social media texts. Finally, these deep parallel contextual representations are post-integrated for emotion prediction. Experiments on a dataset collected from social media regarding community wellness demonstrate that compared to benchmark models, DPCAF achieves at least a 4.81 % increase in Precision, a 3.44 % increase in Recall, and a 10.81 % increase in F1-score. Relative to the most advanced models, DPCAF shows a minimum improvement of 2.65 % in Precision, 3.02 % in Recall, and 2.53 % in F1-score.

Scalable deep learning framework for sentiment analysis prediction for online movie reviews

Sentiment analysis has broad use in diverse real-world contexts, particularly in the online movie industry and other e-commerce platforms. The main objective of our work is to examine the word information order and analyze the content of texts by exploring the hidden meanings of words in online movie text reviews. This study presents an enhanced method of representing text and computationally feasible deep learning models, namely the PEW-MCAB model. The methodology categorizes sentiments by considering the full written text as a unified piece. The feature vector representation is processed using an enhanced text representation called Positional embedding and pretrained Glove Embedding Vector (PEW). The learning of these features is achieved by inculcating a multichannel convolutional neural network (MCNN), which is subsequently integrated into an Attention-based Bidirectional Long Short-Term Memory (AB) model. This experiment examines the positive and negative of online movie textual reviews. Four datasets were used to evaluate the model. When tested on the IMDB, MR (2002), MRC (2004), and MR (2005) datasets, the (PEW-MCAB) algorithm attained accuracy rates of 90.3%, 84.1%, 85.9%, and 87.1%, respectively, in the experimental setting. When implemented in practical settings, the proposed structure shows a great deal of promise for efficacy and competitiveness.

Sentiment Analysis of Social Media Data on Ebola Outbreak Using Deep Learning Classifiers

The Ebola virus disease (EVD) is an extremely contagious and fatal illness caused by the Ebola virus. Recently, Uganda witnessed an outbreak of EVD, which generated much attention on various social media platforms. To ensure effective communication and implementation of targeted health interventions, it is crucial for stakeholders to comprehend the sentiments expressed in the posts and discussions on these online platforms. In this study, we used deep learning techniques to analyse the sentiments expressed in Ebola-related tweets during the outbreak. We explored the application of three deep learning techniques to classify the sentiments in 8395 tweets as positive, neutral, or negative. The techniques examined included a 6-layer convolutional neural network (CNN), a 6-layer long short-term memory model (LSTM), and an 8-layer Bidirectional Encoder Representations from Transformers (BERT) model. The study found that the BERT model outperformed both the CNN and LSTM-based models across all the evaluation metrics, achieving a remarkable classification accuracy of 95%. These findings confirm the reported effectiveness of Transformer-based architectures in tasks related to natural language processing, such as sentiment analysis.

LSA-ac4C: A hybrid neural network incorporating double-layer LSTM and self-attention mechanism for the prediction of N4-acetylcytidine sites in human mRNA

N4-acetylcytidine (ac4C) is a vital constituent of the epitranscriptome and plays a crucial role in the regulation of mRNA expression. Numerous studies have established correlations between ac4C and the incidence, progression and prognosis of various cancers. Therefore, accurately predicting ac4C sites is an important step towards comprehending the biological functions of this modification and devising effective therapeutic interventions. Wet experiments are primary methods for studying ac4C, but computational methods have emerged as a promising supplement due to their cost-effectiveness and shorter research cycles. However, current models still have inherent limitations in terms of predictive performance and generalization ability. Here, we utilized automated machine learning technology to establish a reliable baseline and constructed a deep hybrid neural network, LSA-ac4C, which combines double-layer Long Short-Term Memory (LSTM) and self-attention mechanism for accurate ac4C sites prediction. Benchmarking comparisons demonstrate that LSA-ac4C exhibits superior performance compared to the current state-of-the-art method, with ACC, MCC and AUROC improving by 2.89 %, 5.96 % and 1.53 %, respectively, on an independent test set. Overall, LSA-ac4C serves as a powerful tool for predicting ac4C sites in human mRNA, thus benefiting research on RNA modification. For the convenience of the research community, a web server has been established at http://tubic.org/ac4C.

A Modified Long Short-Term Memory Cell

Machine Learning (ML), among other things, facilitates Text Classification, the task of assigning classes to textual items. Classification performance in ML has been significantly improved due to recent developments, including the rise of Recurrent Neural Networks (RNNs), Long Short-Term Memory (LSTM), Gated Recurrent Units (GRUs), and Transformer Models. Internal memory states with dynamic temporal behavior can be found in these kinds of cells. This temporal behavior in the LSTM cell is stored in two different states: "Current" and "Hidden". In this work, we define a modification layer within the LSTM cell which allows us to perform additional state adjustments for either state, or even simultaneously alter both. We perform 17 state alterations. Out of these 17 single-state alteration experiments, 12 involve the Current state whereas five involve the Hidden one. These alterations are evaluated using seven datasets related to sentiment analysis, document classification, hate speech detection, and human-to-robot interaction. Our results showed that the highest performing alteration for Current and Hidden state can achieve an average F1 improvement of 0.5% and 0.3%, respectively. We also compare our modified cell performance to two Transformer models, where our modified LSTM cell is outperformed in classification metrics in 4/6 datasets, but improves upon the simple Transformer model and clearly has a better cost efficiency than both Transformer models.

Attention-Enabled Ensemble Deep Learning Models and Their Validation for Depression Detection: A Domain Adoption Paradigm

Depression is increasingly prevalent, leading to higher suicide risk. Depression detection and sentimental analysis of text inputs in cross-domain frameworks are challenging. Solo deep learning (SDL) and ensemble deep learning (EDL) models are not robust enough. Recently, attention mechanisms have been introduced in SDL. We hypothesize that attention-enabled EDL (aeEDL) architectures are superior compared to attention-not-enabled SDL (aneSDL) or aeSDL models. We designed EDL-based architectures with attention blocks to build eleven kinds of SDL model and five kinds of EDL model on four domain-specific datasets. We scientifically validated our models by comparing "seen" and "unseen" paradigms (SUP). We benchmarked our results against the SemEval (2016) sentimental dataset and established reliability tests. The mean increase in accuracy for EDL over their corresponding SDL components was 4.49%. Regarding the effect of attention block, the increase in the mean accuracy (AUC) of aeSDL over aneSDL was 2.58% (1.73%), and the increase in the mean accuracy (AUC) of aeEDL over aneEDL was 2.76% (2.80%). When comparing EDL vs. SDL for non-attention and attention, the mean aneEDL was greater than aneSDL by 4.82% (3.71%), and the mean aeEDL was greater than aeSDL by 5.06% (4.81%). For the benchmarking dataset (SemEval), the best-performing aeEDL model (ALBERT+BERT-BiLSTM) was superior to the best aeSDL (BERT-BiLSTM) model by 3.86%. Our scientific validation and robust design showed a difference of only 2.7% in SUP, thereby meeting the regulatory constraints. We validated all our hypotheses and further demonstrated that aeEDL is a very effective and generalized method for detecting symptoms of depression in cross-domain settings.

A deep learning-based model using hybrid feature extraction approach for consumer sentiment analysis

There is an exponential growth in textual content generation every day in today's world. In-app messaging such as Telegram and WhatsApp, social media websites such as Instagram and Facebook, e-commerce websites like Amazon, Google searches, news publishing websites, and a variety of additional sources are the possible suppliers. Every instant, all these sources produce massive amounts of text data. The interpretation of such data can help business owners analyze the social outlook of their product, brand, or service and take necessary steps. The development of a consumer review summarization model using Natural Language Processing (NLP) techniques and Long short-term memory (LSTM) to present summarized data and help businesses obtain substantial insights into their consumers' behavior and choices is the topic of this research. A hybrid approach for analyzing sentiments is presented in this paper. The process comprises pre-processing, feature extraction, and sentiment classification. Using NLP techniques, the pre-processing stage eliminates the undesirable data from input text reviews. For extracting the features effectively, a hybrid method comprising review-related features and aspect-related features has been introduced for constructing the distinctive hybrid feature vector corresponding to each review. The sentiment classification is performed using the deep learning classifier LSTM. We experimentally evaluated the proposed model using three different research datasets. The model achieves the average precision, average recall, and average F1-score of 94.46%, 91.63%, and 92.81%, respectively.

Weighted Joint Sentiment-Topic Model for Sentiment Analysis Compared to ALGA: Adaptive Lexicon Learning Using Genetic Algorithm

Latent Dirichlet Allocation (LDA) is an approach to unsupervised learning that aims to investigate the semantics among words in a document as well as the influence of a subject on a word. As an LDA-based model, Joint Sentiment-Topic (JST) examines the impact of topics and emotions on words. The emotion parameter is insufficient, and additional parameters may play valuable roles in achieving better performance. In this study, two new topic models, Weighted Joint Sentiment-Topic (WJST) and Weighted Joint Sentiment-Topic 1 (WJST1), have been presented to extend and improve JST through two new parameters that can generate a sentiment dictionary. In the proposed methods, each word in a document affects its neighbors, and different words in the document may be affected simultaneously by several neighbor words. Therefore, proposed models consider the effect of words on each other, which, from our view, is an important factor and can increase the performance of baseline methods. Regarding evaluation results, the new parameters have an immense effect on model accuracy. While not requiring labeled data, the proposed methods are more accurate than discriminative models such as SVM and logistic regression in accordance with evaluation results. The proposed methods are simple with a low number of parameters. While providing a broad perception of connections between different words in documents of a single collection (single-domain) or multiple collections (multidomain), the proposed methods have prepared solutions for two different situations (single-domain and multidomain). WJST is suitable for multidomain datasets, and WJST1 is a version of WJST which is suitable for single-domain datasets. While being able to detect emotion at the level of the document, the proposed models improve the evaluation outcomes of the baseline approaches. Thirteen datasets with different sizes have been used in implementations. In this study, perplexity, opinion mining at the level of the document, and topic_coherency are employed for assessment. Also, a statistical test called Friedman test is used to check whether the results of the proposed models are statistically different from the results of other algorithms. As can be seen from results, the accuracy of proposed methods is above 80% for most of the datasets. WJST1 achieves the highest accuracy on Movie dataset with 97 percent, and WJST achieves the highest accuracy on Electronic dataset with 86 percent. The proposed models obtain better results compared to Adaptive Lexicon learning using Genetic Algorithm (ALGA), which employs an evolutionary approach to make an emotion dictionary. Results show that the proposed methods perform better with different topic number settings, especially for WJST1 with 97% accuracy at |Z| = 5 on the Movie dataset.

Integrating Multiclass Light Weighted BiLSTM Model for Classifying Negative Emotions

With the continuous development of social networks, Weibo has become an essential platform for people to share their opinions and feelings in daily life. Analysis of users' emotional tendencies can be effectively applied to public opinion control, public opinion surveys, and product recommendations. However, the traditional deep learning algorithm often needs a large amount of data to be retained to obtain a better accuracy when faced with new work tasks. Given this situation, a multiclassification method of microblog negative sentiment based on MAML (model-agnostic metalearning) and BiLSTM (bidirectional extended short-term memory network) is proposed to represent the microblog text word vectorization and the combination of MAML and BiLSTM is constructed. The model of BiLSTM realizes the classification of negative emotions on Weibo and updates the parameters through machine gradient descent; the metalearner in MAML calculates the sum of the losses of multiple pieces of training, performs a second gradient descent, and updates the metalearner parameters. The updated metalearner can quickly iterate when faced with a new Weibo negative sentiment classification task. The experimental results show that compared with the prepopular model, on the Weibo negative sentiment dataset, the precision rate, recall rate, and F1 value are increased by 1.68 percentage points, 2.86 percentage points, and 2.27 percentage points, respectively.

Document-Level Sentiment Analysis Using Attention-Based Bi-Directional Long Short-Term Memory Network and Two-Dimensional Convolutional Neural Network

Due to outstanding feature extraction ability, neural networks have recently achieved great success in sentiment analysis. However, one of the remaining challenges of sentiment analysis is to model long texts to consider the intrinsic relations between two sentences in the semantic meaning of a document. Moreover, most existing methods are not powerful enough to differentiate the importance of different document features. To address these problems, this paper proposes a new neural network model: AttBiLSTM-2DCNN, which entails two perspectives. First, a two-layer, bidirectional long short-term memory (BiLSTM) network is utilized to obtain the sentiment semantics of a document. The first BiLSTM layer learns the sentiment semantic representation from both directions of a sentence, and the second BiLSTM layer is used to encode the intrinsic relations of sentences into the document matrix representation with a feature dimension and a time-step dimension. Second, a two-dimensional convolutional neural network (2DCNN) is employed to obtain more sentiment dependencies between two sentences. Third, we utilize a two-layer attention mechanism to distinguish the importance of words and sentences in the document. Last, to validate the model, we perform an experiment on two public review datasets that are derived from Yelp2015 and IMDB. Accuracy, F1-Measure, and MSE are used as evaluation metrics. The experimental results show that our model can not only capture sentimental relations but also outperform certain state-of-the-art models.

Sentiment Analysis in Twitter Based on Knowledge Graph and Deep Learning Classification

The traditional way to address the problem of sentiment classification is based on machine learning techniques; however, these models are not able to grasp all the richness of the text that comes from different social media, personal web pages, blogs, etc., ignoring the semantic of the text. Knowledge graphs give a way to extract structured knowledge from images and texts in order to facilitate their semantic analysis. This work proposes a new hybrid approach for Sentiment Analysis based on Knowledge Graphs and Deep Learning techniques to identify the sentiment polarity (positive or negative) in short documents, such as posts on Twitter. In this proposal, tweets are represented as graphs; then, graph similarity metrics and a Deep Learning classification algorithm are applied to produce sentiment predictions. This approach facilitates the traceability and interpretability of the classification results, thanks to the integration of the Local Interpretable Model-agnostic Explanations (LIME) model at the end of the pipeline. LIME allows raising trust in predictive models, since the model is not a black box anymore. Uncovering the black box allows understanding and interpreting how the network could distinguish between sentiment polarities. Each phase of the proposed approach conformed by pre-processing, graph construction, dimensionality reduction, graph similarity, sentiment prediction, and interpretability steps is described. The proposal is compared with character n-gram embeddings-based Deep Learning models to perform Sentiment Analysis. Results show that the proposal is able to outperforms classical n-gram models, with a recall up to 89% and F1-score of 88%.

AI Based Emotion Detection for Textual Big Data: Techniques and Contribution

Online Social Media (OSM) like Facebook and Twitter has emerged as a powerful tool to express via text people’s opinions and feelings about the current surrounding events. Understanding the emotions at the fine-grained level of these expressed thoughts is important for system improvement. Such crucial insights cannot be completely obtained by doing AI-based big data sentiment analysis; hence, text-based emotion detection using AI in social media big data has become an upcoming area of Natural Language Processing research. It can be used in various fields such as understanding expressed emotions, human–computer interaction, data mining, online education, recommendation systems, and psychology. Even though the research work is ongoing in this domain, it still lacks a formal study that can give a qualitative (techniques used) and quantitative (contributions) literature overview. This study has considered 827 Scopus and 83 Web of Science research papers from the years 2005–2020 for the analysis. The qualitative review represents different emotion models, datasets, algorithms, and application domains of text-based emotion detection. The quantitative bibliometric review of contributions presents research details such as publications, volume, co-authorship networks, citation analysis, and demographic research distribution. In the end, challenges and probable solutions are showcased, which can provide future research directions in this area.

Comparative Study of Univariate and Multivariate Long Short-Term Memory for Very Short-Term Forecasting of Global Horizontal Irradiance

Accurate global horizontal irradiance (GHI) forecasting is crucial for efficient management and forecasting of the output power of photovoltaic power plants. However, developing a reliable GHI forecasting model is challenging because GHI varies over time, and its variation is affected by changes in weather patterns. Recently, the long short-term memory (LSTM) deep learning network has become a powerful tool for modeling complex time series problems. This work aims to develop and compare univariate and several multivariate LSTM models that can predict GHI in Guntur, India on a very short-term basis. To build the multivariate time series models, we considered all possible combinations of temperature, humidity, and wind direction variables along with GHI as inputs and developed seven multivariate models, while in the univariate model, we considered only GHI variability. We collected the meteorological data for Guntur from 1 January 2016 to 31 December 2016 and built 12 datasets, each containing variability of GHI, temperature, humidity, and wind direction of a month. We then constructed the models, each of which measures up to 2 h ahead of forecasting of GHI. Finally, to measure the symmetry among the models, we evaluated the performances of the prediction models using root mean square error (RMSE) and mean absolute error (MAE). The results indicate that, compared to the univariate method, each multivariate LSTM performs better in the very short-term GHI prediction task. Moreover, among the multivariate LSTM models, the model that incorporates the temperature variable with GHI as input has outweighed others, achieving average RMSE values 0.74 W/m2–1.5 W/m2.

A novel LSTM-CNN-grid search-based deep neural network for sentiment analysis

As the number of users getting acquainted with the Internet is escalating rapidly, there is more user-generated content on the web. Comprehending hidden opinions, sentiments, and emotions in emails, tweets, reviews, and comments is a challenge and equally crucial for social media monitoring, brand monitoring, customer services, and market research. Sentiment analysis determines the emotional tone behind a series of words may essentially be used to understand the attitude, opinions, and emotions of users. We propose a novel long short-term memory (LSTM)-convolutional neural networks (CNN)-grid search-based deep neural network model for sentiment analysis. The study considers baseline algorithms like convolutional neural networks, K-nearest neighbor, LSTM, neural networks, LSTM-CNN, and CNN-LSTM which have been evaluated using accuracy, precision, sensitivity, specificity, and F-1 score, on multiple datasets. Our results show that the proposed model based on hyperparameter optimization outperforms other baseline models with an overall accuracy greater than 96%.