A hybrid deep-learning approach for complex biochemical named entity recognition

Automatic construction of risk transmission network about subway construction based on deep learning models

Safety risks management is a critical part during the subway construction. However, conventional methods for risk identification heavily rely on experience from experts and fail to effectively identify the relationship between risk factors and events embedded in accident texts, which fail to provide substantial guidance for subway safety risks management. With a dataset comprising 562 occurrences of subway construction accidents, this study devised a domain-specific entity recognition model for identifying safety hazards during the subway construction. The model was constructed by a Bidirectional Long Short-Term Memory Network with Conditional Random Fields (BiLSTM-CRF). Additionally, a domain-specific entity causal relation extraction model employing Convolutional Neural Networks (CNN) was also developed in thsi model. The constructed models automatically extract safety risk factors, safety events, and their causal relationships from the texts about subway accidents. The precision, recall, and F1 scores of Metro Construction Safety Risk Named Entity Recognition Model (MCSR-NER-Model) all exceeded 77%. Its performance in the specialized domain named entity recognition (NER) with a limited volume of textual data is satisfactory. The Metro Construction Safety Risk Domain Entity Causal Relationship Extraction Model (MCSR-CE-Model) achieved an impressive accuracy, recall, and F1 score of 98.96%, exhibiting excellent performance. Moreover, the extracted entities were normalized and domain dictionary was developed. Based on the processed entities and relationships processed by the domain dictionary, 533 domain entity causal relation triplets were obtained, facilitating the establishment of the directed and unweighted complex network and case database about the risks of subway construction. This research successfully converted accident texts into a causal chain structure of "safety risk factors to risk events," providing detailed categorization of safety risks and events. Concurrently, it revealed the interrelationships and historical statistical patterns among various safety risk factors and categories of risk events through the complex safety risks network. The construction of the database facilitated project managers in conducting management decisions about safety risks.

Enhancing Generalizability in Biomedical Entity Recognition: Self-Attention PCA-CLS Model

One of the primary tasks in the early stages of data mining involves the identification of entities from biomedical corpora. Traditional approaches relying on robust feature engineering face challenges when learning from available (un-)annotated data using data-driven models like deep learning-based architectures. Despite leveraging large corpora and advanced deep learning models, domain generalization remains an issue. Attention mechanisms are effective in capturing longer sentence dependencies and extracting semantic and syntactic information from limited annotated datasets. To address out-of-vocabulary challenges in biomedical text, the PCA-CLS (Position and Contextual Attention with CNN-LSTM-Softmax) model combines global self-attention and character-level convolutional neural network techniques. The model's performance is evaluated on eight distinct biomedical domain datasets encompassing entities such as genes, drugs, diseases, and species. The PCA-CLS model outperforms several state-of-the-art models, achieving notable F-scores, including 88.19% on BC2GM, 85.44% on JNLPBA, 90.80% on BC5CDR-chemical, 87.07% on BC5CDR-disease, 89.18% on BC4CHEMD, 88.81% on NCBI, and 91.59% on the s800 dataset.

Construction, evaluation, and application of an electronic medical record corpus for cerebral palsy rehabilitation

Objective

The electronic medical records (EMRs) corpus for cerebral palsy rehabilitation and its application in downstream tasks, such as named entity recognition (NER), requires further revision and testing to enhance its effectiveness and reliability.

Methods

We have devised an annotation principle and have developed an EMRs corpus for cerebral palsy rehabilitation. The introduction of test-retest reliability was employed for the first time to ensure consistency of each annotator. Additionally, we established a baseline NER model using the proposed EMRs corpus. The NER model leveraged Chinese clinical BERT and adversarial training as the embedding layer, and incorporated multi-head attention mechanism and rotary position embedding in the encoder layer. For multi-label decoding, we employed the span matrix of global pointer along with softmax and cross-entropy.

Results

The corpus consisted of 1405 EMRs, containing a total of 127,523 entities across six different entity types, with 24,424 unique entities after de-duplication. The inter-annotator agreement of two annotators was 97.57%, the intra-annotator agreement of each annotator exceeded 98%. Our proposed baseline NER model demonstrates impressive performance, achieving a F1-score of 93.59% for flat entities and 90.15% for nested entities in this corpus.

Conclusions

We believe that the proposed annotation principle, corpus, and baseline model are highly effective and hold great potential as tools for cerebral palsy rehabilitation scenarios.

Impact of word embedding models on text analytics in deep learning environment: a review

The selection of word embedding and deep learning models for better outcomes is vital. Word embeddings are an n-dimensional distributed representation of a text that attempts to capture the meanings of the words. Deep learning models utilize multiple computing layers to learn hierarchical representations of data. The word embedding technique represented by deep learning has received much attention. It is used in various natural language processing (NLP) applications, such as text classification, sentiment analysis, named entity recognition, topic modeling, etc. This paper reviews the representative methods of the most prominent word embedding and deep learning models. It presents an overview of recent research trends in NLP and a detailed understanding of how to use these models to achieve efficient results on text analytics tasks. The review summarizes, contrasts, and compares numerous word embedding and deep learning models and includes a list of prominent datasets, tools, APIs, and popular publications. A reference for selecting a suitable word embedding and deep learning approach is presented based on a comparative analysis of different techniques to perform text analytics tasks. This paper can serve as a quick reference for learning the basics, benefits, and challenges of various word representation approaches and deep learning models, with their application to text analytics and a future outlook on research. It can be concluded from the findings of this study that domain-specific word embedding and the long short term memory model can be employed to improve overall text analytics task performance.

Lexicon and attention-based named entity recognition for kiwifruit diseases and pests: A Deep learning approach

Named Entity Recognition (NER) is a crucial step in mining information from massive agricultural texts, which is required in the construction of many knowledge-based agricultural support systems, such as agricultural technology question answering systems. The vital domain characteristics of Chinese agricultural text cause the Chinese NER (CNER) in kiwifruit diseases and pests to suffer from the insensitivity of common word segmentation tools to kiwifruit-related texts and the feature extraction capability of the sequence encoding layer being challenged. In order to alleviate the above problems, effectively mine information from kiwifruit-related texts to provide support for agricultural support systems such as agricultural question answering systems, this study constructed a novel Chinese agricultural NER (CANER) model KIWINER by statistics-based new word detection and two novel modules, AttSoftlexicon (Criss-cross attention-based Softlexicon) and PCAT (Parallel connection criss-cross attention), proposed in this paper. Specifically, new words were detected to improve the adaptability of word segmentation tools to kiwifruit-related texts, thereby constructing a kiwifruit lexicon. The AttSoftlexicon integrates word information into the model and makes full use of the word information with the help of Criss-cross attention network (CCNet). And the PCAT improves the feature extraction ability of sequence encoding layer through CCNet and parallel connection structure. The performance of KIWINER was evaluated on four datasets, namely KIWID (Self-annotated), Boson, ClueNER, and People's Daily, which achieved optimal F₁-scores of 88.94%, 85.13%, 80.52%, and 92.82%, respectively. Experimental results in many aspects illustrated that methods proposed in this paper can effectively improve the recognition effect of kiwifruit diseases and pests named entities, especially for diseases and pests with strong domain characteristics.

Legal Text Recognition Using LSTM-CRF Deep Learning Model

In legal texts, named entity recognition (NER) is researched using deep learning models. First, the bidirectional (Bi)-long short-term memory (LSTM)-conditional random field (CRF) model for studying NER in legal texts is established. Second, different annotation methods are used to compare and analyze the entity recognition effect of the Bi-LSTM-CRF model. Finally, other objective loss functions are set to compare and analyze the entity recognition effect of the Bi-LSTM-CRF model. The research results show that the F1 value of the model trained on the word sequence labeling corpus on the named entity is 88.13%, higher than that of the word sequence labeling corpus. For the two types of entities, place names and organization names, the F1 values obtained by the Bi-LSTM-CRF model using word segmentation are 67.60% and 89.45%, respectively, higher than the F1 values obtained by the model using character segmentation. Therefore, the Bi-LSTM-CRF model using word segmentation is more suitable for recognizing extended entities. The parameter learning result using log-likelihood is better than that using the maximum interval criterion, and it is ideal for the Bi-LSTM-CRF model. This method provides ideas for the research of legal text recognition and has a particular value.