Transfer Learning for Risk Classification of Social Media Posts: Model Evaluation Study

Development of Internet suicide message identification and the Monitoring-Tracking-Rescuing model in Taiwan

BACKGROUND

Suicide messages can be transmitted infinitely online; the Internet is influential in suicide prevention. Identifying suicide risks online via artificial technological advances may help predict suicide.

METHODS

We built a classifier that detects open messages containing suicidal ideation or behavior-related words in social media via text mining methods and developed the Monitoring-Tracking-Rescuing model, which links data monitoring and tracking to high-risk suicide rescues. Natural language processing (NLP) techniques such as Long Short-Term Memory and Bidirectional Encoder Representations from Transformers were applied to online posts of common social media sites in Taiwan. This model uses a two-step high-risk identification procedure: an automatic prediction process using NLP to classify suicide-risk levels, followed by professional validation by a senior psychiatrist and a nursing faculty specialized in suicidology.

RESULTS

From a dataset containing 404 high-risk and 2226 no- or low-risk articles, the sensitivity and specificity of our model reached 80 %.

LIMITATIONS

The model is limited to data platforms that can be "crawled" and excludes suicide-risk content from graphics, video and audio files. Additionally, machine learning does not provide the best recognition rate from complex online messages. Keywords for high-risk suicide in long articles are difficult to interpret using this model. Finally, the model lacks keywords for suicide-protective factors.

CONCLUSIONS

Artificial intelligence techniques may help detect and monitor high-risk suicide posts and inform mental health professionals of these posts. Periodic tracking plus manual validation to determine risk levels are recommended to enhance the reliability and effectiveness of Internet suicide-prevention tasks.

Smartphone-based artificial intelligence using a transfer learning algorithm for the detection and diagnosis of middle ear diseases: A retrospective deep learning study

BACKGROUND

Middle ear diseases such as otitis media and middle ear effusion, for which diagnoses are often delayed or misdiagnosed, are among the most common issues faced by clinicians providing primary care for children and adolescents. Artificial intelligence (AI) has the potential to assist clinicians in the detection and diagnosis of middle ear diseases through imaging.

METHODS

Otoendoscopic images obtained by otolaryngologists from Taipei Veterans General Hospital in Taiwan between Jany 1, 2011 to Dec 31, 2019 were collected retrospectively and de-identified. The images were entered into convolutional neural network (CNN) training models after data pre-processing, augmentation and splitting. To differentiate sophisticated middle ear diseases, nine CNN-based models were constructed to recognize middle ear diseases. The best-performing models were chosen and ensembled in a small CNN for mobile device use. The pretrained model was converted into the smartphone-based program, and the utility was evaluated in terms of detecting and classifying ten middle ear diseases based on otoendoscopic images. A class activation map (CAM) was also used to identify key features for CNN classification. The performance of each classifier was determined by its accuracy, precision, recall, and F1-score.

FINDINGS

A total of 2820 clinical eardrum images were collected for model training. The programme achieved a high detection accuracy for binary outcomes (pass/refer) of otoendoscopic images and ten different disease categories, with an accuracy reaching 98.0% after model optimisation. Furthermore, the application presented a smooth recognition process and a user-friendly interface and demonstrated excellent performance, with an accuracy of 97.6%. A fifty-question questionnaire related to middle ear diseases was designed for practitioners with different levels of clinical experience. The AI-empowered mobile algorithm's detection accuracy was generally superior to that of general physicians, resident doctors, and otolaryngology specialists (36.0%, 80.0% and 90.0%, respectively). Our results show that the proposed method provides sufficient treatment recommendations that are comparable to those of specialists.

INTERPRETATION

We developed a deep learning model that can detect and classify middle ear diseases. The use of smartphone-based point-of-care diagnostic devices with AI-empowered automated classification can provide real-world smart medical solutions for the diagnosis of middle ear diseases and telemedicine.

FUNDING

This study was supported by grants from the Ministry of Science and Technology (MOST110-2622-8-075-001, MOST110-2320-B-075-004-MY3, MOST-110-2634-F-A49 -005, MOST110-2745-B-075A-001A and MOST110-2221-E-075-005), Veterans General Hospitals and University System of Taiwan Joint Research Program (VGHUST111-G6-11-2, VGHUST111c-140), and Taipei Veterans General Hospital (V111E-002-3).

Detection of Types of Mental Illness through the Social Network Using Ensembled Deep Learning Model

In today's era, social networking platforms are widely used to share emotions. These types of emotions are often analyzed to predict the user's behavior. In this paper, these types of sentiments are classified to predict the mental illness of the user using the ensembled deep learning model. The Reddit social networking platform is used for the analysis, and the ensembling deep learning model is implemented through convolutional neural network and the recurrent neural network. In this work, multiclass classification is performed for predicting mental illness such as anxiety vs. nonanxiety, bipolar vs. nonbipolar, dementia vs. nondementia, and psychotic vs. nonpsychotic. The performance parameters used for evaluating the models are accuracy, precision, recall, and F1 score. The proposed ensemble model used for performing the multiclass classification has performed better than the other models, with an accuracy greater than 92% in predicting the class.

Transfer learning for non-image data in clinical research: A scoping review

BACKGROUND

Transfer learning is a form of machine learning where a pre-trained model trained on a specific task is reused as a starting point and tailored to another task in a different dataset. While transfer learning has garnered considerable attention in medical image analysis, its use for clinical non-image data is not well studied. Therefore, the objective of this scoping review was to explore the use of transfer learning for non-image data in the clinical literature.

METHODS AND FINDINGS

We systematically searched medical databases (PubMed, EMBASE, CINAHL) for peer-reviewed clinical studies that used transfer learning on human non-image data. We included 83 studies in the review. More than half of the studies (63%) were published within 12 months of the search. Transfer learning was most often applied to time series data (61%), followed by tabular data (18%), audio (12%) and text (8%). Thirty-three (40%) studies applied an image-based model to non-image data after transforming data into images (e.g. spectrograms). Twenty-nine (35%) studies did not have any authors with a health-related affiliation. Many studies used publicly available datasets (66%) and models (49%), but fewer shared their code (27%).

CONCLUSIONS

In this scoping review, we have described current trends in the use of transfer learning for non-image data in the clinical literature. We found that the use of transfer learning has grown rapidly within the last few years. We have identified studies and demonstrated the potential of transfer learning in clinical research in a wide range of medical specialties. More interdisciplinary collaborations and the wider adaption of reproducible research principles are needed to increase the impact of transfer learning in clinical research.

Transfer Learning with Social Media Content in the Ride-Hailing Domain by Using a Hybrid Machine Learning Architecture

The analysis of the content of posts written on social media has established an important line of research in recent years. The study of these texts, as well as their relationship with each other and their dependence on the platform on which they are written, enables the behavior analysis of users and their opinions with respect to different domains. In this work, a hybrid machine learning-based system has been developed to classify texts using topic modeling techniques and different word-vector representations, as well as traditional text representations. The system has been trained with ride-hailing posts extracted from Reddit, showing promising performance. Then, the generated models have been tested with data extracted from other sources such as Twitter and Google Play, classifying these texts without retraining any models and thus performing Transfer Learning. The obtained results show that our proposed architecture is effective when performing Transfer Learning from data-rich domains and applying them to other sources.

Natural language processing in clinical neuroscience and psychiatry: A review

Natural language processing (NLP) is rapidly becoming an important topic in the medical community. The ability to automatically analyze any type of medical document could be the key factor to fully exploit the data it contains. Cutting-edge artificial intelligence (AI) architectures, particularly machine learning and deep learning, have begun to be applied to this topic and have yielded promising results. We conducted a literature search for 1,024 papers that used NLP technology in neuroscience and psychiatry from 2010 to early 2022. After a selection process, 115 papers were evaluated. Each publication was classified into one of three categories: information extraction, classification, and data inference. Automated understanding of clinical reports in electronic health records has the potential to improve healthcare delivery. Overall, the performance of NLP applications is high, with an average F1-score and AUC above 85%. We also derived a composite measure in the form of Z-scores to better compare the performance of NLP models and their different classes as a whole. No statistical differences were found in the unbiased comparison. Strong asymmetry between English and non-English models, difficulty in obtaining high-quality annotated data, and train biases causing low generalizability are the main limitations. This review suggests that NLP could be an effective tool to help clinicians gain insights from medical reports, clinical research forms, and more, making NLP an effective tool to improve the quality of healthcare services.

Detecting risk level in individuals misusing fentanyl utilizing posts from an online community on Reddit

INTRODUCTION

Opioid misuse is a public health crisis in the US, and misuse of synthetic opioids such as fentanyl have driven the most recent waves of opioid-related deaths. Because those who misuse fentanyl are often a hidden and high-risk group, innovative methods for identifying individuals at risk for fentanyl misuse are needed. Machine learning has been used in the past to investigate discussions surrounding substance use on Reddit, and this study leverages similar techniques to identify risky content from discussions of fentanyl on this platform.

METHODS

A codebook was developed by clinical domain experts with 12 categories indicative of fentanyl misuse risk, and this was used to manually label 391 Reddit posts and comments. Using this data, we built machine learning classification models to identify fentanyl risk.

RESULTS

Our machine learning risk model was able to detect posts or comments labeled as risky by our clinical experts with 76% accuracy and 76% sensitivity. Furthermore, we provide a vocabulary of community-specific, colloquial words for fentanyl and its analogues.

DISCUSSION

This study uses an interdisciplinary approach leveraging machine learning techniques and clinical domain expertise to automatically detect risky discourse, which may elicit and benefit from timely intervention. Moreover, our vocabulary of online terms for fentanyl and its analogues expands our understanding of online "street" nomenclature for opiates. Through an improved understanding of substance misuse risk factors, these findings allow for identification of risk concepts among those misusing fentanyl to inform outreach and intervention strategies tailored to this at-risk group.

The promise of automated machine learning for the genetic analysis of complex traits

The genetic analysis of complex traits has been dominated by parametric statistical methods due to their theoretical properties, ease of use, computational efficiency, and intuitive interpretation. However, there are likely to be patterns arising from complex genetic architectures which are more easily detected and modeled using machine learning methods. Unfortunately, selecting the right machine learning algorithm and tuning its hyperparameters can be daunting for experts and non-experts alike. The goal of automated machine learning (AutoML) is to let a computer algorithm identify the right algorithms and hyperparameters thus taking the guesswork out of the optimization process. We review the promises and challenges of AutoML for the genetic analysis of complex traits and give an overview of several approaches and some example applications to omics data. It is our hope that this review will motivate studies to develop and evaluate novel AutoML methods and software in the genetics and genomics space. The promise of AutoML is to enable anyone, regardless of training or expertise, to apply machine learning as part of their genetic analysis strategy.