Machine Learning and Natural Language Processing in Mental Health: Systematic Review

Interdisciplinary approach to identify language markers for post-traumatic stress disorder using machine learning and deep learning

Post-traumatic stress disorder (PTSD) lacks clear biomarkers in clinical practice. Language as a potential diagnostic biomarker for PTSD is investigated in this study. We analyze an original cohort of 148 individuals exposed to the November 13, 2015, terrorist attacks in Paris. The interviews, conducted 5-11 months after the event, include individuals from similar socioeconomic backgrounds exposed to the same incident, responding to identical questions and using uniform PTSD measures. Using this dataset to collect nuanced insights that might be clinically relevant, we propose a three-step interdisciplinary methodology that integrates expertise from psychiatry, linguistics, and the Natural Language Processing (NLP) community to examine the relationship between language and PTSD. The first step assesses a clinical psychiatrist's ability to diagnose PTSD using interview transcription alone. The second step uses statistical analysis and machine learning models to create language features based on psycholinguistic hypotheses and evaluate their predictive strength. The third step is the application of a hypothesis-free deep learning approach to the classification of PTSD in our cohort. Results show that the clinical psychiatrist achieved a diagnosis of PTSD with an AUC of 0.72. This is comparable to a gold standard questionnaire (Area Under Curve (AUC) ≈ 0.80). The machine learning model achieved a diagnostic AUC of 0.69. The deep learning approach achieved an AUC of 0.64. An examination of model error informs our discussion. Importantly, the study controls for confounding factors, establishes associations between language and DSM-5 subsymptoms, and integrates automated methods with qualitative analysis. This study provides a direct and methodologically robust description of the relationship between PTSD and language. Our work lays the groundwork for advancing early and accurate diagnosis and using linguistic markers to assess the effectiveness of pharmacological treatments and psychotherapies.

AI Quality Standards in Health Care: Rapid Umbrella Review

BACKGROUND

In recent years, there has been an upwelling of artificial intelligence (AI) studies in the health care literature. During this period, there has been an increasing number of proposed standards to evaluate the quality of health care AI studies.

OBJECTIVE

This rapid umbrella review examines the use of AI quality standards in a sample of health care AI systematic review articles published over a 36-month period.

METHODS

We used a modified version of the Joanna Briggs Institute umbrella review method. Our rapid approach was informed by the practical guide by Tricco and colleagues for conducting rapid reviews. Our search was focused on the MEDLINE database supplemented with Google Scholar. The inclusion criteria were English-language systematic reviews regardless of review type, with mention of AI and health in the abstract, published during a 36-month period. For the synthesis, we summarized the AI quality standards used and issues noted in these reviews drawing on a set of published health care AI standards, harmonized the terms used, and offered guidance to improve the quality of future health care AI studies.

RESULTS

We selected 33 review articles published between 2020 and 2022 in our synthesis. The reviews covered a wide range of objectives, topics, settings, designs, and results. Over 60 AI approaches across different domains were identified with varying levels of detail spanning different AI life cycle stages, making comparisons difficult. Health care AI quality standards were applied in only 39% (13/33) of the reviews and in 14% (25/178) of the original studies from the reviews examined, mostly to appraise their methodological or reporting quality. Only a handful mentioned the transparency, explainability, trustworthiness, ethics, and privacy aspects. A total of 23 AI quality standard-related issues were identified in the reviews. There was a recognized need to standardize the planning, conduct, and reporting of health care AI studies and address their broader societal, ethical, and regulatory implications.

CONCLUSIONS

Despite the growing number of AI standards to assess the quality of health care AI studies, they are seldom applied in practice. With increasing desire to adopt AI in different health topics, domains, and settings, practitioners and researchers must stay abreast of and adapt to the evolving landscape of health care AI quality standards and apply these standards to improve the quality of their AI studies.

Exploring the Role of Artificial Intelligence in Mental Healthcare: Current Trends and Future Directions - A Narrative Review for a Comprehensive Insight

Mental health is an essential component of the health and well-being of a person and community, and it is critical for the individual, society, and socio-economic development of any country. Mental healthcare is currently in the health sector transformation era, with emerging technologies such as artificial intelligence (AI) reshaping the screening, diagnosis, and treatment modalities of psychiatric illnesses. The present narrative review is aimed at discussing the current landscape and the role of AI in mental healthcare, including screening, diagnosis, and treatment. Furthermore, this review attempted to highlight the key challenges, limitations, and prospects of AI in providing mental healthcare based on existing works of literature. The literature search for this narrative review was obtained from PubMed, Saudi Digital Library (SDL), Google Scholar, Web of Science, and IEEE Xplore, and we included only English-language articles published in the last five years. Keywords used in combination with Boolean operators ("AND" and "OR") were the following: "Artificial intelligence", "Machine learning", Deep learning", "Early diagnosis", "Treatment", "interventions", "ethical consideration", and "mental Healthcare". Our literature review revealed that, equipped with predictive analytics capabilities, AI can improve treatment planning by predicting an individual's response to various interventions. Predictive analytics, which uses historical data to formulate preventative interventions, aligns with the move toward individualized and preventive mental healthcare. In the screening and diagnostic domains, a subset of AI, such as machine learning and deep learning, has been proven to analyze various mental health data sets and predict the patterns associated with various mental health problems. However, limited studies have evaluated the collaboration between healthcare professionals and AI in delivering mental healthcare, as these sensitive problems require empathy, human connections, and holistic, personalized, and multidisciplinary approaches. Ethical issues, cybersecurity, a lack of data analytics diversity, cultural sensitivity, and language barriers remain concerns for implementing this futuristic approach in mental healthcare. Considering these sensitive problems require empathy, human connections, and holistic, personalized, and multidisciplinary approaches, it is imperative to explore these aspects. Therefore, future comparative trials with larger sample sizes and data sets are warranted to evaluate different AI models used in mental healthcare across regions to fill the existing knowledge gaps.

Using Large Language Models to Understand Suicidality in a Social Media-Based Taxonomy of Mental Health Disorders: Linguistic Analysis of Reddit Posts

Background

Rates of suicide have increased by over 35% since 1999. Despite concerted efforts, our ability to predict, explain, or treat suicide risk has not significantly improved over the past 50 years.

Objective

The aim of this study was to use large language models to understand natural language use during public web-based discussions (on Reddit) around topics related to suicidality.

Methods

We used large language model-based sentence embedding to extract the latent linguistic dimensions of user postings derived from several mental health-related subreddits, with a focus on suicidality. We then applied dimensionality reduction to these sentence embeddings, allowing them to be summarized and visualized in a lower-dimensional Euclidean space for further downstream analyses. We analyzed 2.9 million posts extracted from 30 subreddits, including r/SuicideWatch, between October 1 and December 31, 2022, and the same period in 2010.

Results

Our results showed that, in line with existing theories of suicide, posters in the suicidality community (r/SuicideWatch) predominantly wrote about feelings of disconnection, burdensomeness, hopeless, desperation, resignation, and trauma. Further, we identified distinct latent linguistic dimensions (well-being, seeking support, and severity of distress) among all mental health subreddits, and many of the resulting subreddit clusters were in line with a statistically driven diagnostic classification system-namely, the Hierarchical Taxonomy of Psychopathology (HiTOP)-by mapping onto the proposed superspectra.

Conclusions

Overall, our findings provide data-driven support for several language-based theories of suicide, as well as dimensional classification systems for mental health disorders. Ultimately, this novel combination of natural language processing techniques can assist researchers in gaining deeper insights about emotions and experiences shared on the web and may aid in the validation and refutation of different mental health theories.

Leveraging Novel Technologies and Artificial Intelligence to Advance Practice-Oriented Research

Mental health services are experiencing notable transformations as innovative technologies and artificial intelligence (AI) are increasingly utilized in a growing number of studies and services.These cutting-edge technologies carry the promise of substantial improvements in the field of mental health. Nevertheless, questions emerge about the alignment of novel technologies and AI systems with human needs, especially in the context of vulnerable populations receiving mental healthcare. The practice-oriented research (POR) model is pivotal in seamlessly integrating these emerging technologies into clinical research and practice. It underscores the importance of tight collaboration between clinicians and researchers, all driven by the central goal of ensuring and elevating client well-being. This paper focuses on how novel technologies can enhance the POR model and highlights its pivotal role in integrating these technologies into clinical research and practice. We discuss two key phases: pre-treatment, and during treatment. For each phase, we describe the challenges, present the major technological innovations, describe recent studies exemplifying technology use, and suggest future directions. Ethical concerns and the importance of aligning humans and technology are also considered, in addition to implications for practice and training.

Conversational assessment using artificial intelligence is as clinically useful as depression scales and preferred by users

BACKGROUND

Depression is prevalent, chronic, and burdensome. Due to limited screening access, depression often remains undiagnosed. Artificial intelligence (AI) models based on spoken responses to interview questions may offer an effective, efficient alternative to other screening methods.

OBJECTIVE

The primary aim was to use a demographically diverse sample to validate an AI model, previously trained on human-administered interviews, on novel bot-administered interviews, and to check for algorithmic biases related to age, sex, race, and ethnicity.

METHODS

Using the Aiberry app, adults recruited via social media (N = 393) completed a brief bot-administered interview and a depression self-report form. An AI model was used to predict form scores based on interview responses alone. For all meaningful discrepancies between model inference and form score, clinicians performed a masked review to determine which one they preferred.

RESULTS

There was strong concurrent validity between the model predictions and raw self-report scores (r = 0.73, MAE = 3.3). 90 % of AI predictions either agreed with self-report or with clinical expert opinion when AI contradicted self-report. There was no differential model performance across age, sex, race, or ethnicity.

LIMITATIONS

Limitations include access restrictions (English-speaking ability and access to smartphone or computer with broadband internet) and potential self-selection of participants more favorably predisposed toward AI technology.

CONCLUSION

The Aiberry model made accurate predictions of depression severity based on remotely collected spoken responses to a bot-administered interview. This study shows promising results for the use of AI as a mental health screening tool on par with self-report measures.

Using natural language processing to analyze unstructured patient-reported outcomes data derived from electronic health records for cancer populations: a systematic review

INTRODUCTION

Patient-reported outcomes (PROs; symptoms, functional status, quality-of-life) expressed in the 'free-text' or 'unstructured' format within clinical notes from electronic health records (EHRs) offer valuable insights beyond biological and clinical data for medical decision-making. However, a comprehensive assessment of utilizing natural language processing (NLP) coupled with machine learning (ML) methods to analyze unstructured PROs and their clinical implementation for individuals affected by cancer remains lacking.

AREAS COVERED

This study aimed to systematically review published studies that used NLP techniques to extract and analyze PROs in clinical narratives from EHRs for cancer populations. We examined the types of NLP (with and without ML) techniques and platforms for data processing, analysis, and clinical applications.

EXPERT OPINION

Utilizing NLP methods offers a valuable approach for processing and analyzing unstructured PROs among cancer patients and survivors. These techniques encompass a broad range of applications, such as extracting or recognizing PROs, categorizing, characterizing, or grouping PROs, predicting or stratifying risk for unfavorable clinical results, and evaluating connections between PROs and adverse clinical outcomes. The employment of NLP techniques is advantageous in converting substantial volumes of unstructured PRO data within EHRs into practical clinical utilities for individuals with cancer.

Bridging information gaps in menopause status classification through natural language processing

Objective

To use natural language processing (NLP) of clinical notes to augment existing structured electronic health record (EHR) data for classification of a patient's menopausal status.

Materials and methods

A rule-based NLP system was designed to capture evidence of a patient's menopause status including dates of a patient's last menstrual period, reproductive surgeries, and postmenopause diagnosis as well as their use of birth control and menstrual interruptions. NLP-derived output was used in combination with structured EHR data to classify a patient's menopausal status. NLP processing and patient classification were performed on a cohort of 307 512 female Veterans receiving healthcare at the US Department of Veterans Affairs (VA).

Results

NLP was validated at 99.6% precision. Including the NLP-derived data into a menopause phenotype increased the number of patients with data relevant to their menopausal status by 118%. Using structured codes alone, 81 173 (27.0%) are able to be classified as postmenopausal or premenopausal. However, with the inclusion of NLP, this number increased 167 804 (54.6%) patients. The premenopausal category grew by 532.7% with the inclusion of NLP data.

Discussion

By employing NLP, it became possible to identify documented data elements that predate VA care, originate outside VA networks, or have no corresponding structured field in the VA EHR that would be otherwise inaccessible for further analysis.

Conclusion

NLP can be used to identify concepts relevant to a patient's menopausal status in clinical notes. Adding NLP-derived data to an algorithm classifying a patient's menopausal status significantly increases the number of patients classified using EHR data, ultimately enabling more detailed assessments of the impact of menopause on health outcomes.

Healthcare Systems and Artificial Intelligence: Focus on Challenges and the International Regulatory Framework

BACKGROUND

Nowadays, healthcare systems are coping with the challenge of countering the exponential growth of healthcare costs worldwide, to support sustainability and to guarantee access to treatment for all patients.

METHODS

Artificial Intelligence (AI) is the technology able to perform human cognitive functions through the creation of algorithms. The value of AI in healthcare and its ability to address healthcare delivery issues has been a subject of discussion within the scientific community for several years.

RESULTS

The aim of this work is to provide an overview of the primary uses of AI in the healthcare system, to discuss its desirable future uses while shedding light on the major issues related to implications within international regulatory processes. In this manuscript, it will be described the main applications of AI in various aspects of health care, from clinical studies to ethical implications, focusing on the international regulatory framework in countries in which AI is used, to discuss and compare strengthens and weaknesses.

CONCLUSIONS

The challenges in regulatory processes to facilitate the integration of AI in healthcare are significant. However, overcoming them is essential to ensure that AI-based technologies are adopted safely and effectively.

Uncovering lobbying strategies in sustainable finance disclosure regulations using machine learning

PURPOSE

We analyse lobbying behaviour by using Machine Learning approaches. In the context of Sustainable Finance Disclosure Regulation (SFDR), we gain detailed insights, assign these to existing strategies, and measure how strongly which participant influences the regulation.

STUDY DESIGN/METHODOLOGY/APPROACH

We use tri-gram analysis, sentiment analysis, and similarity analysis as methods to obtain insights into the political commentary process of European Supervisory Authorities (ESAs) drafts dealing with SFDR.

FINDINGS

Our metadata helps to identify stakeholders and lobbying strategies. We found that the most negative comments came from the regulated, who argued strongly subjectively in a very objective environment of ESG disclosure. We also identified typical lobbying strategies based on arguments, persuasion, and classic cost-benefit considerations.

ORIGINALITY/VALUE

We generate emotion values and synthesise detailed argument differences and show that modern algorithms can contribute to the identification of interest groups and lobbying strategies. Furthermore, we generate similarity values of arguments that can be taken into account in the analysis of the success of a lobbying strategy.

SADXAI: Predicting social anxiety disorder using multiple interpretable artificial intelligence techniques

Social anxiety disorder (SAD), also known as social phobia, is a psychological condition in which a person has a persistent and overwhelming fear of being negatively judged or observed by other individuals. This fear can affect them at work, in relationships and other social activities. The intricate combination of several environmental and biological factors is the reason for the onset of this mental condition. SAD is diagnosed using a test called the "Diagnostic and Statistical Manual of Mental Health Disorders (DSM-5), which is based on several physical, emotional and demographic symptoms. Artificial Intelligence has been a boon for medicine and is regularly used to diagnose various health conditions and diseases. Hence, this study used demographic, emotional, and physical symptoms and multiple machine learning (ML) techniques to diagnose SAD. A thorough descriptive and statistical analysis has been conducted before using the classifiers. Among all the models, the AdaBoost and logistic regression obtained the highest accuracy of 88 % each. Four eXplainable artificial techniques (XAI) techniques are utilized to make the predictions interpretable, transparent and understandable. According to XAI, the "Liebowitz Social Anxiety Scale questionnaire" and "The fear of speaking in public" are the most critical attributes in the diagnosis of SAD. This clinical decision support system framework could be utilized in various suitable locations such as schools, hospitals and workplaces to identify SAD in people.

Automated machine learning-based model for the prediction of pedicle screw loosening after degenerative lumbar fusion surgery

The adequacy of screw anchorage is a critical factor in achieving successful spinal fusion. This study aimed to use machine learning algorithms to identify critical variables and predict pedicle screw loosening after degenerative lumbar fusion surgery. A total of 552 patients who underwent primary transpedicular lumbar fixation for lumbar degenerative disease were included. The LASSO method identified key features associated with pedicle screw loosening. Patient clinical characteristics, intraoperative variables, and radiographic parameters were collected and used to construct eight machine learning models, including a training set (80% of participants) and a test set (20% of participants). The XGBoost model exhibited the best performance, with an AUC of 0.884 (95% CI: 0.825-0.944) in the test set, along with the lowest Brier score. Ten crucial variables, including age, disease diagnosis: degenerative scoliosis, number of fused levels, fixation to S1, HU value, preoperative PT, preoperative PI-LL, postoperative LL, postoperative PT, and postoperative PI-LL were selected. In the prospective cohort, the XGBoost model demonstrated substantial performance with an accuracy of 83.32%. This study identified crucial variables associated with pedicle screw loosening after degenerative lumbar fusion surgery and successfully developed a machine learning model to predict pedicle screw loosening. The findings of this study may provide valuable information for clinical decision-making.

Deep learning-based automatic segmentation of meningioma from T1-weighted contrast-enhanced MRI for preoperative meningioma differentiation using radiomic features

BACKGROUND

This study aimed to establish a dedicated deep-learning model (DLM) on routine magnetic resonance imaging (MRI) data to investigate DLM performance in automated detection and segmentation of meningiomas in comparison to manual segmentations. Another purpose of our work was to develop a radiomics model based on the radiomics features extracted from automatic segmentation to differentiate low- and high-grade meningiomas before surgery.

MATERIALS

A total of 326 patients with pathologically confirmed meningiomas were enrolled. Samples were randomly split with a 6:2:2 ratio to the training set, validation set, and test set. Volumetric regions of interest (VOIs) were manually drawn on each slice using the ITK-SNAP software. An automatic segmentation model based on SegResNet was developed for the meningioma segmentation. Segmentation performance was evaluated by dice coefficient and 95% Hausdorff distance. Intra class correlation (ICC) analysis was applied to assess the agreement between radiomic features from manual and automatic segmentations. Radiomics features derived from automatic segmentation were extracted by pyradiomics. After feature selection, a model for meningiomas grading was built.

RESULTS

The DLM detected meningiomas in all cases. For automatic segmentation, the mean dice coefficient and 95% Hausdorff distance were 0.881 (95% CI: 0.851-0.981) and 2.016 (95% CI:1.439-3.158) in the test set, respectively. Features extracted on manual and automatic segmentation are comparable: the average ICC value was 0.804 (range, 0.636-0.933). Features extracted on manual and automatic segmentation are comparable: the average ICC value was 0.804 (range, 0.636-0.933). For meningioma classification, the radiomics model based on automatic segmentation performed well in grading meningiomas, yielding a sensitivity, specificity, accuracy, and area under the curve (AUC) of 0.778 (95% CI: 0.701-0.856), 0.860 (95% CI: 0.722-0.908), 0.848 (95% CI: 0.715-0.903) and 0.842 (95% CI: 0.807-0.895) in the test set, respectively.

CONCLUSIONS

The DLM yielded favorable automated detection and segmentation of meningioma and can help deploy radiomics for preoperative meningioma differentiation in clinical practice.

Using artificial intelligence for exercise prescription in personalised health promotion: A critical evaluation of OpenAI's GPT-4 model

The rise of artificial intelligence (AI) applications in healthcare provides new possibilities for personalized health management. AI-based fitness applications are becoming more common, facilitating the opportunity for individualised exercise prescription. However, the use of AI carries the risk of inadequate expert supervision, and the efficacy and validity of such applications have not been thoroughly investigated, particularly in the context of diverse health conditions. The aim of the study was to critically assess the efficacy of exercise prescriptions generated by OpenAI's Generative Pre-Trained Transformer 4 (GPT-4) model for five example patient profiles with diverse health conditions and fitness goals. Our focus was to assess the model's ability to generate exercise prescriptions based on a singular, initial interaction, akin to a typical user experience. The evaluation was conducted by leading experts in the field of exercise prescription. Five distinct scenarios were formulated, each representing a hypothetical individual with a specific health condition and fitness objective. Upon receiving details of each individual, the GPT-4 model was tasked with generating a 30-day exercise program. These AI-derived exercise programs were subsequently subjected to a thorough evaluation by experts in exercise prescription. The evaluation encompassed adherence to established principles of frequency, intensity, time, and exercise type; integration of perceived exertion levels; consideration for medication intake and the respective medical condition; and the extent of program individualization tailored to each hypothetical profile. The AI model could create general safety-conscious exercise programs for various scenarios. However, the AI-generated exercise prescriptions lacked precision in addressing individual health conditions and goals, often prioritizing excessive safety over the effectiveness of training. The AI-based approach aimed to ensure patient improvement through gradual increases in training load and intensity, but the model's potential to fine-tune its recommendations through ongoing interaction was not fully satisfying. AI technologies, in their current state, can serve as supplemental tools in exercise prescription, particularly in enhancing accessibility for individuals unable to access, often costly, professional advice. However, AI technologies are not yet recommended as a substitute for personalized, progressive, and health condition-specific prescriptions provided by healthcare and fitness professionals. Further research is needed to explore more interactive use of AI models and integration of real-time physiological feedback.

The Role of Artificial Intelligence in Identifying Depression and Anxiety: A Comprehensive Literature Review

This narrative literature review undertakes a comprehensive examination of the burgeoning field, tracing the development of artificial intelligence (AI)-powered tools for depression and anxiety detection from the level of intricate algorithms to practical applications. Delivering essential mental health care services is now a significant public health priority. In recent years, AI has become a game-changer in the early identification and intervention of these pervasive mental health disorders. AI tools can potentially empower behavioral healthcare services by helping psychiatrists collect objective data on patients' progress and tasks. This study emphasizes the current understanding of AI, the different types of AI, its current use in multiple mental health disorders, advantages, disadvantages, and future potentials. As technology develops and the digitalization of the modern era increases, there will be a rise in the application of artificial intelligence in psychiatry; therefore, a comprehensive understanding will be needed. We searched PubMed, Google Scholar, and Science Direct using keywords for this. In a recent review of studies using electronic health records (EHR) with AI and machine learning techniques for diagnosing all clinical conditions, roughly 99 publications have been found. Out of these, 35 studies were identified for mental health disorders in all age groups, and among them, six studies utilized EHR data sources. By critically analyzing prominent scholarly works, we aim to illuminate the current state of this technology, exploring its successes, limitations, and future directions. In doing so, we hope to contribute to a nuanced understanding of AI's potential to revolutionize mental health diagnostics and pave the way for further research and development in this critically important domain.

Digital accessibility in the era of artificial intelligence-Bibliometric analysis and systematic review

Introduction

Digital accessibility involves designing digital systems and services to enable access for individuals, including those with disabilities, including visual, auditory, motor, or cognitive impairments. Artificial intelligence (AI) has the potential to enhance accessibility for people with disabilities and improve their overall quality of life.

Methods

This systematic review, covering academic articles from 2018 to 2023, focuses on AI applications for digital accessibility. Initially, 3,706 articles were screened from five scholarly databases-ACM Digital Library, IEEE Xplore, ScienceDirect, Scopus, and Springer.

Results

The analysis narrowed down to 43 articles, presenting a classification framework based on applications, challenges, AI methodologies, and accessibility standards.

Discussion

This research emphasizes the predominant focus on AI-driven digital accessibility for visual impairments, revealing a critical gap in addressing speech and hearing impairments, autism spectrum disorder, neurological disorders, and motor impairments. This highlights the need for a more balanced research distribution to ensure equitable support for all communities with disabilities. The study also pointed out a lack of adherence to accessibility standards in existing systems, stressing the urgency for a fundamental shift in designing solutions for people with disabilities. Overall, this research underscores the vital role of accessible AI in preventing exclusion and discrimination, urging a comprehensive approach to digital accessibility to cater to diverse disability needs.

Evaluating Large Language Models in Extracting Cognitive Exam Dates and Scores

Importance

Large language models (LLMs) are crucial for medical tasks. Ensuring their reliability is vital to avoid false results. Our study assesses two state-of-the-art LLMs (ChatGPT and LlaMA-2) for extracting clinical information, focusing on cognitive tests like MMSE and CDR.

Objective

Evaluate ChatGPT and LlaMA-2 performance in extracting MMSE and CDR scores, including their associated dates.

Methods

Our data consisted of 135,307 clinical notes (Jan 12th, 2010 to May 24th, 2023) mentioning MMSE, CDR, or MoCA. After applying inclusion criteria 34,465 notes remained, of which 765 underwent ChatGPT (GPT-4) and LlaMA-2, and 22 experts reviewed the responses. ChatGPT successfully extracted MMSE and CDR instances with dates from 742 notes. We used 20 notes for fine-tuning and training the reviewers. The remaining 722 were assigned to reviewers, with 309 each assigned to two reviewers simultaneously. Inter-rater-agreement (Fleiss' Kappa), precision, recall, true/false negative rates, and accuracy were calculated. Our study follows TRIPOD reporting guidelines for model validation.

Results

For MMSE information extraction, ChatGPT (vs. LlaMA-2) achieved accuracy of 83% (vs. 66.4%), sensitivity of 89.7% (vs. 69.9%), true-negative rates of 96% (vs 60.0%), and precision of 82.7% (vs 62.2%). For CDR the results were lower overall, with accuracy of 87.1% (vs. 74.5%), sensitivity of 84.3% (vs. 39.7%), true-negative rates of 99.8% (98.4%), and precision of 48.3% (vs. 16.1%). We qualitatively evaluated the MMSE errors of ChatGPT and LlaMA-2 on double-reviewed notes. LlaMA-2 errors included 27 cases of total hallucination, 19 cases of reporting other scores instead of MMSE, 25 missed scores, and 23 cases of reporting only the wrong date. In comparison, ChatGPT's errors included only 3 cases of total hallucination, 17 cases of wrong test reported instead of MMSE, and 19 cases of reporting a wrong date.

Conclusions

In this diagnostic/prognostic study of ChatGPT and LlaMA-2 for extracting cognitive exam dates and scores from clinical notes, ChatGPT exhibited high accuracy, with better performance compared to LlaMA-2. The use of LLMs could benefit dementia research and clinical care, by identifying eligible patients for treatments initialization or clinical trial enrollments. Rigorous evaluation of LLMs is crucial to understanding their capabilities and limitations.

Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

As mental health (MH) disorders become increasingly prevalent, their multifaceted symptoms and comorbidities with other conditions introduce complexity to diagnosis, posing a risk of underdiagnosis. While machine learning (ML) has been explored to mitigate these challenges, we hypothesized that multiple data modalities support more comprehensive detection and that non-intrusive collection approaches better capture natural behaviors. To understand the current trends, we systematically reviewed 184 studies to assess feature extraction, feature fusion, and ML methodologies applied to detect MH disorders from passively sensed multimodal data, including audio and video recordings, social media, smartphones, and wearable devices. Our findings revealed varying correlations of modality-specific features in individualized contexts, potentially influenced by demographics and personalities. We also observed the growing adoption of neural network architectures for model-level fusion and as ML algorithms, which have demonstrated promising efficacy in handling high-dimensional features while modeling within and cross-modality relationships. This work provides future researchers with a clear taxonomy of methodological approaches to multimodal detection of MH disorders to inspire future methodological advancements. The comprehensive analysis also guides and supports future researchers in making informed decisions to select an optimal data source that aligns with specific use cases based on the MH disorder of interest.

Systematic reviews of machine learning in healthcare: a literature review

INTRODUCTION

The increasing availability of data and computing power has made machine learning (ML) a viable approach to faster, more efficient healthcare delivery.

METHODS

A systematic literature review (SLR) of published SLRs evaluating ML applications in healthcare settings published between1 January 2010 and 27 March 2023 was conducted.

RESULTS

In total 220 SLRs covering 10,462 ML algorithms were reviewed. The main application of AI in medicine related to the clinical prediction and disease prognosis in oncology and neurology with the use of imaging data. Accuracy, specificity, and sensitivity were provided in 56%, 28%, and 25% SLRs respectively. Internal and external validation was reported in 53% and less than 1% of the cases respectively. The most common modeling approach was neural networks (2,454 ML algorithms), followed by support vector machine and random forest/decision trees (1,578 and 1,522 ML algorithms, respectively).

EXPERT OPINION

The review indicated considerable reporting gaps in terms of the ML's performance, both internal and external validation. Greater accessibility to healthcare data for developers can ensure the faster adoption of ML algorithms into clinical practice.

Predicting Patients' Satisfaction With Mental Health Drug Treatment Using Their Reviews: Unified Interchangeable Model Fusion Approach

BACKGROUND

After the COVID-19 pandemic, the conflict between limited mental health care resources and the rapidly growing number of patients has become more pronounced. It is necessary for psychologists to borrow artificial intelligence (AI)-based methods to analyze patients' satisfaction with drug treatment for those undergoing mental illness treatment.

OBJECTIVE

Our goal was to construct highly accurate and transferable models for predicting the satisfaction of patients with mental illness with medication by analyzing their own experiences and comments related to medication intake.

METHODS

We extracted 41,851 reviews in 20 categories of disorders related to mental illnesses from a large public data set of 161,297 reviews in 16,950 illness categories. To discover a more optimal structure of the natural language processing models, we proposed the Unified Interchangeable Model Fusion to decompose the state-of-the-art Bidirectional Encoder Representations from Transformers (BERT), support vector machine, and random forest (RF) models into 2 modules, the encoder and the classifier, and then reconstruct fused "encoder+classifer" models to accurately evaluate patients' satisfaction. The fused models were divided into 2 categories in terms of model structures, traditional machine learning-based models and neural network-based models. A new loss function was proposed for those neural network-based models to overcome overfitting and data imbalance. Finally, we fine-tuned the fused models and evaluated their performance comprehensively in terms of F1-score, accuracy, κ coefficient, and training time using 10-fold cross-validation.

RESULTS

Through extensive experiments, the transformer bidirectional encoder+RF model outperformed the state-of-the-art BERT, MentalBERT, and other fused models. It became the optimal model for predicting the patients' satisfaction with drug treatment. It achieved an average graded F1-score of 0.872, an accuracy of 0.873, and a κ coefficient of 0.806. This model is suitable for high-standard users with sufficient computing resources. Alternatively, it turned out that the word-embedding encoder+RF model showed relatively good performance with an average graded F1-score of 0.801, an accuracy of 0.812, and a κ coefficient of 0.695 but with much less training time. It can be deployed in environments with limited computing resources.

CONCLUSIONS

We analyzed the performance of support vector machine, RF, BERT, MentalBERT, and all fused models and identified the optimal models for different clinical scenarios. The findings can serve as evidence to support that the natural language processing methods can effectively assist psychologists in evaluating the satisfaction of patients with drug treatment programs and provide precise and standardized solutions. The Unified Interchangeable Model Fusion provides a different perspective on building AI models in mental health and has the potential to fuse the strengths of different components of the models into a single model, which may contribute to the development of AI in mental health.

Large Language Models in Neurology Research and Future Practice

Recent advancements in generative artificial intelligence, particularly using large language models (LLMs), are gaining increased public attention. We provide a perspective on the potential of LLMs to analyze enormous amounts of data from medical records and gain insights on specific topics in neurology. In addition, we explore use cases for LLMs, such as early diagnosis, supporting patient and caregivers, and acting as an assistant for clinicians. We point to the potential ethical and technical challenges raised by LLMs, such as concerns about privacy and data security, potential biases in the data for model training, and the need for careful validation of results. Researchers must consider these challenges and take steps to address them to ensure that their work is conducted in a safe and responsible manner. Despite these challenges, LLMs offer promising opportunities for improving care and treatment of various neurologic disorders.

Detection of Suicidal Ideation in Clinical Interviews for Depression Using Natural Language Processing and Machine Learning: Cross-Sectional Study

Background

Assessing patients' suicide risk is challenging, especially among those who deny suicidal ideation. Primary care providers have poor agreement in screening suicide risk. Patients' speech may provide more objective, language-based clues about their underlying suicidal ideation. Text analysis to detect suicide risk in depression is lacking in the literature.

Objective

This study aimed to determine whether suicidal ideation can be detected via language features in clinical interviews for depression using natural language processing (NLP) and machine learning (ML).

Methods

This cross-sectional study recruited 305 participants between October 2020 and May 2022 (mean age 53.0, SD 11.77 years; female: n=176, 57%), of which 197 had lifetime depression and 108 were healthy. This study was part of ongoing research on characterizing depression with a case-control design. In this study, 236 participants were nonsuicidal, while 56 and 13 had low and high suicide risks, respectively. The structured interview guide for the Hamilton Depression Rating Scale (HAMD) was adopted to assess suicide risk and depression severity. Suicide risk was clinician rated based on a suicide-related question (H11). The interviews were transcribed and the words in participants' verbal responses were translated into psychologically meaningful categories using Linguistic Inquiry and Word Count (LIWC).

Results

Ordinal logistic regression revealed significant suicide-related language features in participants' responses to the HAMD questions. Increased use of anger words when talking about work and activities posed the highest suicide risk (odds ratio [OR] 2.91, 95% CI 1.22-8.55; P=.02). Random forest models demonstrated that text analysis of the direct responses to H11 was effective in identifying individuals with high suicide risk (AUC 0.76-0.89; P<.001) and detecting suicide risk in general, including both low and high suicide risk (AUC 0.83-0.92; P<.001). More importantly, suicide risk can be detected with satisfactory performance even without patients' disclosure of suicidal ideation. Based on the response to the question on hypochondriasis, ML models were trained to identify individuals with high suicide risk (AUC 0.76; P<.001).

Conclusions

This study examined the perspective of using NLP and ML to analyze the texts from clinical interviews for suicidality detection, which has the potential to provide more accurate and specific markers for suicidal ideation detection. The findings may pave the way for developing high-performance assessment of suicide risk for automated detection, including online chatbot-based interviews for universal screening.

Opening the Black Box of Family-Based Treatments: An Artificial Intelligence Framework to Examine Therapeutic Alliance and Therapist Empathy

The evidence-based treatment (EBT) movement has primarily focused on core intervention content or treatment fidelity and has largely ignored practitioner skills to manage interpersonal process issues that emerge during treatment, especially with difficult-to-treat adolescents (delinquent, substance-using, medical non-adherence) and those of color. A chief complaint of "real world" practitioners about manualized treatments is the lack of correspondence between following a manual and managing microsocial interpersonal processes (e.g. negative affect) that arise in treating "real world clients." Although family-based EBTs share core similarities (e.g. focus on family interactions, emphasis on practitioner engagement, family involvement), most of these treatments do not have an evidence base regarding common implementation and treatment process problems that practitioners experience in delivering particular models, especially in mid-treatment when demands on families to change their behavior is greatest in treatment - a lack that characterizes the field as a whole. Failure to effectively address common interpersonal processes with difficult-to-treat families likely undermines treatment fidelity and sustained use of EBTs, treatment outcome, and contributes to treatment dropout and treatment nonadherence. Recent advancements in wearables, sensing technologies, multivariate time-series analyses, and machine learning allow scientists to make significant advancements in the study of psychotherapy processes by looking "under the skin" of the provider-client interpersonal interactions that define therapeutic alliance, empathy, and empathic accuracy, along with the predictive validity of these therapy processes (therapeutic alliance, therapist empathy) to treatment outcome. Moreover, assessment of these processes can be extended to develop procedures for training providers to manage difficult interpersonal processes while maintaining a physiological profile that is consistent with astute skills in psychotherapeutic processes. This paper argues for opening the "black box" of therapy to advance the science of evidence-based psychotherapy by examining the clinical interior of evidence-based treatments to develop the next generation of audit- and feedback- (i.e., systemic review of professional performance) supervision systems.

Finding the Best Match - a Case Study on the (Text-)Feature and Model Choice in Digital Mental Health Interventions

With the need for psychological help long exceeding the supply, finding ways of scaling, and better allocating mental health support is a necessity. This paper contributes by investigating how to best predict intervention dropout and failure to allow for a need-based adaptation of treatment. We systematically compare the predictive power of different text representation methods (metadata, TF-IDF, sentiment and topic analysis, and word embeddings) in combination with supplementary numerical inputs (socio-demographic, evaluation, and closed-question data). Additionally, we address the research gap of which ML model types - ranging from linear to sophisticated deep learning models - are best suited for different features and outcome variables. To this end, we analyze nearly 16.000 open-text answers from 849 German-speaking users in a Digital Mental Health Intervention (DMHI) for stress. Our research proves that - contrary to previous findings - there is great promise in using neural network approaches on DMHI text data. We propose a task-specific LSTM-based model architecture to tackle the challenge of long input sequences and thereby demonstrate the potential of word embeddings (AUC scores of up to 0.7) for predictions in DMHIs. Despite the relatively small data set, sequential deep learning models, on average, outperform simpler features such as metadata and bag-of-words approaches when predicting dropout. The conclusion is that user-generated text of the first two sessions carries predictive power regarding patients' dropout and intervention failure risk. Furthermore, the match between the sophistication of features and models needs to be closely considered to optimize results, and additional non-text features increase prediction results.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00148-z.

Identifying self-reported health-related problems in home-based rehabilitation of older patients after hip replacement in China: a machine learning study based on Omaha system theory

BACKGROUND

With the aging of the population, the number of total hip replacement surgeries is increasing globally. Hip replacement has undergone revolutionary advancements in surgical methods and materials. Due to the short length of hospitalization, rehabilitation care is mainly home-based. The needs and concerns about such home-based rehabilitation are constantly changing, requiring continuous attention.

OBJECTIVE

To explore effective methods for comprehensively identifying older patients' self-reported outcomes after home-based rehabilitation for hip replacement, in order to develop appropriate intervention strategies for patient rehabilitation care in the future.

METHODS

This study constructed a corpus of patients' self-reported rehabilitation care problems after hip replacement, based on the Omaha classification system. This study used the Python development language and implemented artificial intelligence to match the corpus data on the cooperation platform, to identify the main health-related problems reported by the patients, and to perform statistical analyses.

RESULTS

Most patients had physical health-related problems. More than 80% of these problems were related to neuromusculoskeletal function, interpersonal relationships, pain, health care supervision, physical activity, vision, nutrition, and residential environment. The most common period in which patients' self-reported problems arose was 6 months post-surgery. The relevant labels that were moderately related to these problems were: Physiology-Speech and Language and Physiology-Mind (r = 0.45), Health-Related Behaviors-Nutrition and Health-Related Behaviors-Compliance with Doctors' Prescription (r = 0.40).

CONCLUSION

Physiological issues remain the main health-related issues for home-based rehabilitation after hip replacement in older patients. Precision care has become an important principle of rehabilitation care. This study used a machine learning method to obtain the largest quantitative network data possible. The artificial intelligence capture was fully automated, which greatly improved efficiency, as compared to manual data entering.

Reduced Order Machine Learning Models for Accurate Prediction of CO2 Capture in Physical Solvents

CO2 sorption in physical solvents is one of the promising approaches for carbon capture from highly concentrated CO2 streams at high pressures. Identifying an efficient solvent and evaluating its solubility data at different operating conditions are highly essential for effective capture, which generally involves expensive and time-consuming experimental procedures. This work presents a machine learning based ultrafast alternative for accurate prediction of CO2 solubility in physical solvents using their physical, thermodynamic, and structural properties data. First, a database is established with which several linear, nonlinear, and ensemble models were trained through a systematic cross-validation and grid search method and found that kernel ridge regression (KRR) is the optimum model. Second, the descriptors are ranked based on their complete decomposition contributions derived using principal component analysis. Further, optimum key descriptors (KDs) are evaluated through an iterative sequential addition method with the objective of maximizing the prediction accuracy of the reduced order KRR (r-KRR) model. Finally, the study resulted in the r-KRR model with nine KDs exhibiting the highest prediction accuracy with a minimum root-mean-square error (0.0023), mean absolute error (0.0016), and maximum R2 (0.999). Also, the validity of the database created and ML models developed is ensured through detailed statistical analysis.

What can we learn from a Chinese social media used by glaucoma patients?

PURPOSE

Our study aims to discuss glaucoma patients' needs and Internet habits using big data analysis and Natural Language Processing (NLP) based on deep learning (DL).

METHODS

In this retrospective study, we used web crawler technology to crawl glaucoma-related topic posts from the glaucoma bar of Baidu Tieba, China. According to the contents of topic posts, we classified them into posts with seeking medical advice and without seeking medical advice (social support, expressing emotions, sharing knowledge, and others). Word Cloud and frequency statistics were used to analyze the contents and visualize the keywords of topic posts. Two DL models, Bidirectional Long Short-Term Memory (Bi-LSTM) and Bidirectional Encoder Representations from Transformers (BERT), were trained to identify the posts seeking medical advice. The evaluation matrices included: accuracy, F1 value, and the area under the ROC curve (AUC).

RESULTS

A total of 10,892 topic posts were included, among them, most were seeking medical advice (N = 7071, 64.91%), and seeking advice regarding symptoms or examination (N = 4913, 45.11%) dominated the majority. The following were searching for social support (N = 2362, 21.69%), expressing emotions (N = 497, 4.56%), and sharing knowledge (N = 527, 4.84%) in sequence. The word cloud analysis results showed that ocular pressure, visual field, examination, and operation were the most frequent words. The accuracy, F1 score, and AUC were 0.891, 0.891, and 0.931 for the BERT model, 0.82, 0.821, and 0.890 for the Bi-LSTM model.

CONCLUSION

Social media can help enhance the patient-doctor relationship by providing patients' concerns and cognition about glaucoma in China. NLP can be a powerful tool to reflect patients' focus on diseases. DL models performed well in classifying Chinese medical-related texts, which could play an important role in public health monitoring.

Trustworthy Data and AI Environments for Clinical Prediction: Application to Crisis-Risk in People With Depression

Depression is a common mental health condition that often occurs in association with other chronic illnesses, and varies considerably in severity. Electronic Health Records (EHRs) contain rich information about a patient's medical history and can be used to train, test and maintain predictive models to support and improve patient care. This work evaluated the feasibility of implementing an environment for predicting mental health crisis among people living with depression based on both structured and unstructured EHRs. A large EHR from a mental health provider, Mersey Care, was pseudonymised and ingested into the Natural Language Processing (NLP) platform CogStack, allowing text content in binary clinical notes to be extracted. All unstructured clinical notes and summaries were semantically annotated by MedCAT and BioYODIE NLP services. Cases of crisis in patients with depression were then identified. Random forest models, gradient boosting trees, and Long Short-Term Memory (LSTM) networks, with varying feature arrangement, were trained to predict the occurrence of crisis. The results showed that all the prediction models can use a combination of structured and unstructured EHR information to predict crisis in patients with depression with good and useful accuracy. The LSTM network that was trained on a modified dataset with only 1000 most-important features from the random forest model with temporality showed the best performance with a mean AUC of 0.901 and a standard deviation of 0.006 using a training dataset and a mean AUC of 0.810 and 0.01 using a hold-out test dataset. Comparing the results from the technical evaluation with the views of psychiatrists shows that there are now opportunities to refine and integrate such prediction models into pragmatic point-of-care clinical decision support tools for supporting mental healthcare delivery.

Extracting cancer concepts from clinical notes using natural language processing: a systematic review

BACKGROUND

Extracting information from free texts using natural language processing (NLP) can save time and reduce the hassle of manually extracting large quantities of data from incredibly complex clinical notes of cancer patients. This study aimed to systematically review studies that used NLP methods to identify cancer concepts from clinical notes automatically.

METHODS

PubMed, Scopus, Web of Science, and Embase were searched for English language papers using a combination of the terms concerning "Cancer", "NLP", "Coding", and "Registries" until June 29, 2021. Two reviewers independently assessed the eligibility of papers for inclusion in the review.

RESULTS

Most of the software programs used for concept extraction reported were developed by the researchers (n = 7). Rule-based algorithms were the most frequently used algorithms for developing these programs. In most articles, the criteria of accuracy (n = 14) and sensitivity (n = 12) were used to evaluate the algorithms. In addition, Systematized Nomenclature of Medicine-Clinical Terms (SNOMED-CT) and Unified Medical Language System (UMLS) were the most commonly used terminologies to identify concepts. Most studies focused on breast cancer (n = 4, 19%) and lung cancer (n = 4, 19%).

CONCLUSION

The use of NLP for extracting the concepts and symptoms of cancer has increased in recent years. The rule-based algorithms are well-liked algorithms by developers. Due to these algorithms' high accuracy and sensitivity in identifying and extracting cancer concepts, we suggested that future studies use these algorithms to extract the concepts of other diseases as well.

Investigating the Readability and Linguistic, Psychological, and Emotional Characteristics of Digital Dementia Information Written in the English Language: Multitrait-Multimethod Text Analysis

BACKGROUND

Past research in the Western context found that people with dementia search for digital dementia information in peer-reviewed medical research articles, dementia advocacy and medical organizations, and blogs written by other people with dementia. This past work also demonstrated that people with dementia do not perceive English digital dementia information as emotionally or cognitively accessible.

OBJECTIVE

In this study, we sought to investigate the readability; linguistic, psychological, and emotional characteristics; and target audiences of digital dementia information. We conducted a textual analysis of 3 different types of text-based digital dementia information written in English: 300 medical articles, 35 websites, and 50 blogs.

METHODS

We assessed the text's readability using the Flesch Reading Ease and Flesch-Kincaid Grade Level measurements, as well as tone, analytical thinking, clout, authenticity, and word frequencies using a natural language processing tool, Linguistic Inquiry and Word Count Generator. We also conducted a thematic analysis to categorize the target audiences for each information source and used these categorizations for further statistical analysis.

RESULTS

The median Flesch-Kincaid Grade Level readability score and Flesch Reading Ease score for all types of information (N=1139) were 12.1 and 38.6, respectively, revealing that the readability scores of all 3 information types were higher than the minimum requirement. We found that medical articles had significantly (P=.05) higher word count and analytical thinking scores as well as significantly lower clout, authenticity, and emotional tone scores than websites and blogs. Further, blogs had significantly (P=.48) higher word count and authenticity scores but lower analytical scores than websites. Using thematic analysis, we found that most of the blogs (156/227, 68.7%) and web pages (399/612, 65.2%) were targeted at people with dementia. Website information targeted at a general audience had significantly lower readability scores. In addition, website information targeted at people with dementia had higher word count and lower emotional tone ratings. The information on websites targeted at caregivers had significantly higher clout and lower authenticity scores.

CONCLUSIONS

Our findings indicate that there is an abundance of digital dementia information written in English that is targeted at people with dementia, but this information is not readable by a general audience. This is problematic considering that people with <12 years of education are at a higher risk of developing dementia. Further, our findings demonstrate that digital dementia information written in English has a negative tone, which may be a contributing factor to the mental health crisis many people with dementia face after receiving a diagnosis. Therefore, we call for content creators to lower readability scores to make the information more accessible to a general audience and to focus their efforts on providing information in a way that does not perpetuate overly negative narratives of dementia.

Artificial intelligence-driven approach for patient-focused drug development

Patients' increasing digital participation provides an opportunity to pursue patient-centric research and drug development by understanding their needs. Social media has proven to be one of the most useful data sources when it comes to understanding a company's potential audience to drive more targeted impact. Navigating through an ocean of information is a tedious task where techniques such as artificial intelligence and text analytics have proven effective in identifying relevant posts for healthcare business questions. Here, we present an enterprise-ready, scalable solution demonstrating the feasibility and utility of social media-based patient experience data for use in research and development through capturing and assessing patient experiences and expectations on disease, treatment options, and unmet needs while creating a playbook for roll-out to other indications and therapeutic areas.

Natural language processing analysis of the psychosocial stressors of mental health disorders during the pandemic

Over the past few years, the COVID-19 pandemic has exerted various impacts on the world, notably concerning mental health. Nevertheless, the precise influence of psychosocial stressors on this mental health crisis remains largely unexplored. In this study, we employ natural language processing to examine chat text from a mental health helpline. The data was obtained from a chat helpline called Safe Hour from the "It Gets Better" project in Chile. This dataset encompass 10,986 conversations between trained professional volunteers from the foundation and platform users from 2018 to 2020. Our analysis shows a significant increase in conversations covering issues of self-image and interpersonal relations, as well as a decrease in performance themes. Also, we observe that conversations involving themes like self-image and emotional crisis played a role in explaining both suicidal behavior and depressive symptoms. However, anxious symptoms can only be explained by emotional crisis themes. These findings shed light on the intricate connections between psychosocial stressors and various mental health aspects in the context of the COVID-19 pandemic.

Detecting depression tendency with multimodal features

BACKGROUND AND OBJECTIVE

Depression can severely impact physical and mental health and may even harm society. Therefore, detecting the early symptoms of depression and treating them on time is critical. The widespread use of social media has led individuals with depressive tendencies to express their emotions on social platforms, share their painful experiences, and seek support and help. Therefore, the massive available amounts of social platform data provide the possibility of identifying depressive tendencies.

METHODS

This paper proposes a neural network hybrid model MTDD to achieve this goal. Analysis of the content of users' posts on social platforms has facilitated constructing a post-level method to detect depressive tendencies in individuals. Compared with existing methods, the MTDD model uses the following innovative methods: First, this model is based on social platform data, which is objective and accurate, can be obtained at a low cost, and is easy to operate. The model can avoid the influence of subjective factors in the depressive tendency detection method based on consultation with mental health experts. In other words, it can avoid the problem of undisclosed and imperfect data in depressive tendency detection. Second, the MTDD model is based on a deep neural network hybrid model, combining the advantages of CNN and BiLSTM networks and avoiding the problem of poor generalization ability in a single model for depression tendency recognition. Third, the MTDD model is based on multimodal features for learning the vector representation of depression-prone text, including text features, semantic features, and domain knowledge, making the model more robust.

RESULTS

Extensive experimental results demonstrate that our MTDD model detects users who may have a depressive tendency with a 95% F1 value and obtained SOTA results.

CONCLUSIONS

Our MTDD model can detect depressive users on social media platforms more effectively, providing the possibility for early diagnosis and timely treatment of depression. The experiment proves that our MTDD model outperforms many of the latest depressive tendency detection models.

Machine learning models for predicting the risk factor of carotid plaque in cardiovascular disease

Introduction

Cardiovascular disease (CVD) is a group of diseases involving the heart or blood vessels and represents a leading cause of death and disability worldwide. Carotid plaque is an important risk factor for CVD that can reflect the severity of atherosclerosis. Accordingly, developing a prediction model for carotid plaque formation is essential to assist in the early prevention and management of CVD.

Methods

In this study, eight machine learning algorithms were established, and their performance in predicting carotid plaque risk was compared. Physical examination data were collected from 4,659 patients and used for model training and validation. The eight predictive models based on machine learning algorithms were optimized using the above dataset and 10-fold cross-validation. The Shapley Additive Explanations (SHAP) tool was used to compute and visualize feature importance. Then, the performance of the models was evaluated according to the area under the receiver operating characteristic curve (AUC), feature importance, accuracy and specificity.

Results

The experimental results indicated that the XGBoost algorithm outperformed the other machine learning algorithms, with an AUC, accuracy and specificity of 0.808, 0.749 and 0.762, respectively. Moreover, age, smoke, alcohol drink and BMI were the top four predictors of carotid plaque formation. It is feasible to predict carotid plaque risk using machine learning algorithms.

Conclusions

This study indicates that our models can be applied to routine chronic disease management procedures to enable more preemptive, broad-based screening for carotid plaque and improve the prognosis of CVD patients.

A Call to Action on Assessing and Mitigating Bias in Artificial Intelligence Applications for Mental Health

Advances in computer science and data-analytic methods are driving a new era in mental health research and application. Artificial intelligence (AI) technologies hold the potential to enhance the assessment, diagnosis, and treatment of people experiencing mental health problems and to increase the reach and impact of mental health care. However, AI applications will not mitigate mental health disparities if they are built from historical data that reflect underlying social biases and inequities. AI models biased against sensitive classes could reinforce and even perpetuate existing inequities if these models create legacies that differentially impact who is diagnosed and treated, and how effectively. The current article reviews the health-equity implications of applying AI to mental health problems, outlines state-of-the-art methods for assessing and mitigating algorithmic bias, and presents a call to action to guide the development of fair-aware AI in psychological science.

Generalizability and portability of natural language processing system to extract individual social risk factors

OBJECTIVE

The objective of this study is to validate and report on portability and generalizability of a Natural Language Processing (NLP) method to extract individual social factors from clinical notes, which was originally developed at a different institution.

MATERIALS AND METHODS

A rule-based deterministic state machine NLP model was developed to extract financial insecurity and housing instability using notes from one institution and was applied on all notes written during 6 months at another institution. 10% of positively-classified notes by NLP and the same number of negatively-classified notes were manually annotated. The NLP model was adjusted to accommodate notes at the new site. Accuracy, positive predictive value, sensitivity, and specificity were calculated.

RESULTS

More than 6 million notes were processed at the receiving site by the NLP model, which resulted in about 13,000 and 19,000 classified as positive for financial insecurity and housing instability, respectively. The NLP model showed excellent performance on the validation dataset with all measures over 0.87 for both social factors.

DISCUSSION

Our study illustrated the need to accommodate institution-specific note-writing templates as well as clinical terminology of emergent diseases when applying NLP model for social factors. A state machine is relatively simple to port effectively across institutions. Our study. showed superior performance to similar generalizability studies for extracting social factors.

CONCLUSION

Rule-based NLP model to extract social factors from clinical notes showed strong portability and generalizability across organizationally and geographically distinct institutions. With only relatively simple modifications, we obtained promising performance from an NLP-based model.

Diagnostic and Management Performance of ChatGPT in Obstetrics and Gynecology

OBJECTIVES

The use of artificial intelligence (AI) in clinical patient management and medical education has been advancing over time. ChatGPT was developed and trained recently, using a large quantity of textual data from the internet. Medical science is expected to be transformed by its use. The present study was conducted to evaluate the diagnostic and management performance of the ChatGPT AI model in obstetrics and gynecology.

DESIGN

A cross-sectional study was conducted.

PARTICIPANTS/MATERIALS, SETTING, METHODS

This study was conducted in Iran in March 2023. Medical histories and examination results of 30 cases were determined in six areas of obstetrics and gynecology. The cases were presented to a gynecologist and ChatGPT for diagnosis and management. Answers from the gynecologist and ChatGPT were compared, and the diagnostic and management performance of ChatGPT were determined.

RESULTS

Ninety percent (27 of 30) of the cases in obstetrics and gynecology were correctly handled by ChatGPT. Its responses were eloquent, informed, and free of a significant number of errors or misinformation. Even when the answers provided by ChatGPT were incorrect, the responses contained a logical explanation about the case as well as information provided in the question stem.

LIMITATIONS

The data used in this study were taken from the electronic book and may reflect bias in the diagnosis of ChatGPT.

CONCLUSIONS

This is the first evaluation of ChatGPT's performance in diagnosis and management in the field of obstetrics and gynecology. It appears that ChatGPT has potential applications in the practice of medicine and is (currently) free and simple to use. However, several ethical considerations and limitations such as bias, validity, copyright infringement, and plagiarism need to be addressed in future studies.

Natural language processing-driven state machines to extract social factors from unstructured clinical documentation

Objective

This study sought to create natural language processing algorithms to extract the presence of social factors from clinical text in 3 areas: (1) housing, (2) financial, and (3) unemployment. For generalizability, finalized models were validated on data from a separate health system for generalizability.

Materials and Methods

Notes from 2 healthcare systems, representing a variety of note types, were utilized. To train models, the study utilized n-grams to identify keywords and implemented natural language processing (NLP) state machines across all note types. Manual review was conducted to determine performance. Sampling was based on a set percentage of notes, based on the prevalence of social need. Models were optimized over multiple training and evaluation cycles. Performance metrics were calculated using positive predictive value (PPV), negative predictive value, sensitivity, and specificity.

Results

PPV for housing rose from 0.71 to 0.95 over 3 training runs. PPV for financial rose from 0.83 to 0.89 over 2 training iterations, while PPV for unemployment rose from 0.78 to 0.88 over 3 iterations. The test data resulted in PPVs of 0.94, 0.97, and 0.95 for housing, financial, and unemployment, respectively. Final specificity scores were 0.95, 0.97, and 0.95 for housing, financial, and unemployment, respectively.

Discussion

We developed 3 rule-based NLP algorithms, trained across health systems. While this is a less sophisticated approach, the algorithms demonstrated a high degree of generalizability, maintaining >0.85 across all predictive performance metrics.

Conclusion

The rule-based NLP algorithms demonstrated consistent performance in identifying 3 social factors within clinical text. These methods may be a part of a strategy to measure social factors within an institution.

Enhancing an enterprise data warehouse for research with data extracted using natural language processing

Objective

This study aims to develop a generalizable architecture for enhancing an enterprise data warehouse for research (EDW4R) with results from a natural language processing (NLP) model, which allows discrete data derived from clinical notes to be made broadly available for research use without need for NLP expertise. The study also quantifies the additional value that information extracted from clinical narratives brings to EDW4R.

Materials and methods

Clinical notes written during one month at an academic health center were used to evaluate the performance of an existing NLP model and to quantify its value added to the structured data. Manual review was utilized for performance analysis. The architecture for enhancing the EDW4R is described in detail to enable reproducibility.

Results

Two weeks were needed to enhance EDW4R with data from 250 million clinical notes. NLP generated 16 and 39% increase in data availability for two variables.

Discussion

Our architecture is highly generalizable to a new NLP model. The positive predictive value obtained by an independent team showed only slightly lower NLP performance than the values reported by the NLP developers. The NLP showed significant value added to data already available in structured format.

Conclusion

Given the value added by data extracted using NLP, it is important to enhance EDW4R with these data to enable research teams without NLP expertise to benefit from value added by NLP models.

Virtually screening adults for depression, anxiety, and suicide risk using machine learning and language from an open-ended interview

Background

Current depression, anxiety, and suicide screening techniques rely on retrospective patient reported symptoms to standardized scales. A qualitative approach to screening combined with the innovation of natural language processing (NLP) and machine learning (ML) methods have shown promise to enhance person-centeredness while detecting depression, anxiety, and suicide risk from in-the-moment patient language derived from an open-ended brief interview.

Objective

To evaluate the performance of NLP/ML models to identify depression, anxiety, and suicide risk from a single 5-10-min semi-structured interview with a large, national sample.

Method

Two thousand four hundred sixteen interviews were conducted with 1,433 participants over a teleconference platform, with 861 (35.6%), 863 (35.7%), and 838 (34.7%) sessions screening positive for depression, anxiety, and suicide risk, respectively. Participants completed an interview over a teleconference platform to collect language about the participants' feelings and emotional state. Logistic regression (LR), support vector machine (SVM), and extreme gradient boosting (XGB) models were trained for each condition using term frequency-inverse document frequency features from the participants' language. Models were primarily evaluated with the area under the receiver operating characteristic curve (AUC).

Results

The best discriminative ability was found when identifying depression with an SVM model (AUC = 0.77; 95% CI = 0.75-0.79), followed by anxiety with an LR model (AUC = 0.74; 95% CI = 0.72-0.76), and an SVM for suicide risk (AUC = 0.70; 95% CI = 0.68-0.72). Model performance was generally best with more severe depression, anxiety, or suicide risk. Performance improved when individuals with lifetime but no suicide risk in the past 3 months were considered controls.

Conclusion

It is feasible to use a virtual platform to simultaneously screen for depression, anxiety, and suicide risk using a 5-to-10-min interview. The NLP/ML models performed with good discrimination in the identification of depression, anxiety, and suicide risk. Although the utility of suicide risk classification in clinical settings is still undetermined and suicide risk classification had the lowest performance, the result taken together with the qualitative responses from the interview can better inform clinical decision-making by providing additional drivers associated with suicide risk.

The emergent role of artificial intelligence, natural learning processing, and large language models in higher education and research

Artificial Intelligence (AI) has revolutionized various domains, including education and research. Natural language processing (NLP) techniques and large language models (LLMs) such as GPT-4 and BARD have significantly advanced our comprehension and application of AI in these fields. This paper provides an in-depth introduction to AI, NLP, and LLMs, discussing their potential impact on education and research. By exploring the advantages, challenges, and innovative applications of these technologies, this review gives educators, researchers, students, and readers a comprehensive view of how AI could shape educational and research practices in the future, ultimately leading to improved outcomes. Key applications discussed in the field of research include text generation, data analysis and interpretation, literature review, formatting and editing, and peer review. AI applications in academics and education include educational support and constructive feedback, assessment, grading, tailored curricula, personalized career guidance, and mental health support. Addressing the challenges associated with these technologies, such as ethical concerns and algorithmic biases, is essential for maximizing their potential to improve education and research outcomes. Ultimately, the paper aims to contribute to the ongoing discussion about the role of AI in education and research and highlight its potential to lead to better outcomes for students, educators, and researchers.

Methodologies for Monitoring Mental Health on Twitter: Systematic Review

BACKGROUND

The use of social media data to predict mental health outcomes has the potential to allow for the continuous monitoring of mental health and well-being and provide timely information that can supplement traditional clinical assessments. However, it is crucial that the methodologies used to create models for this purpose are of high quality from both a mental health and machine learning perspective. Twitter has been a popular choice of social media because of the accessibility of its data, but access to big data sets is not a guarantee of robust results.

OBJECTIVE

This study aims to review the current methodologies used in the literature for predicting mental health outcomes from Twitter data, with a focus on the quality of the underlying mental health data and the machine learning methods used.

METHODS

A systematic search was performed across 6 databases, using keywords related to mental health disorders, algorithms, and social media. In total, 2759 records were screened, of which 164 (5.94%) papers were analyzed. Information about methodologies for data acquisition, preprocessing, model creation, and validation was collected, as well as information about replicability and ethical considerations.

RESULTS

The 164 studies reviewed used 119 primary data sets. There were an additional 8 data sets identified that were not described in enough detail to include, and 6.1% (10/164) of the papers did not describe their data sets at all. Of these 119 data sets, only 16 (13.4%) had access to ground truth data (ie, known characteristics) about the mental health disorders of social media users. The other 86.6% (103/119) of data sets collected data by searching keywords or phrases, which may not be representative of patterns of Twitter use for those with mental health disorders. The annotation of mental health disorders for classification labels was variable, and 57.1% (68/119) of the data sets had no ground truth or clinical input on this annotation. Despite being a common mental health disorder, anxiety received little attention.

CONCLUSIONS

The sharing of high-quality ground truth data sets is crucial for the development of trustworthy algorithms that have clinical and research utility. Further collaboration across disciplines and contexts is encouraged to better understand what types of predictions will be useful in supporting the management and identification of mental health disorders. A series of recommendations for researchers in this field and for the wider research community are made, with the aim of enhancing the quality and utility of future outputs.

Detection of Depression-Related Tweets in Mexico Using Crosslingual Schemes and Knowledge Distillation

Mental health problems are one of the various ills that afflict the world’s population. Early diagnosis and medical care are public health problems addressed from various perspectives. Among the mental illnesses that most afflict the population is depression; its early diagnosis is vitally important, as it can trigger more severe illnesses, such as suicidal ideation. Due to the lack of homogeneity in current diagnostic tools, the community has focused on using AI tools for opportune diagnosis. Unfortunately, there is a lack of data that allows the use of IA tools for the Spanish language. Our work has a cross-lingual scheme to address this issue, allowing us to identify Spanish and English texts. The experiments demonstrated the methodology’s effectiveness with an F1-score of 0.95. With this methodology, we propose a method to solve a classification problem for depression tweets (or short texts) by reusing English language databases with insufficient data to generate a classification model, such as in the Spanish language. We also validated the information obtained with public data to analyze the behavior of depression in Mexico during the COVID-19 pandemic. Our results show that the use of these methodologies can serve as support, not only in the diagnosis of depression, but also in the construction of different language databases that allow the creation of more efficient diagnostic tools.

A Unified Training Process for Fake News Detection Based on Finetuned Bidirectional Encoder Representation from Transformers Model

An efficient fake news detector becomes essential as the accessibility of social media platforms increases rapidly. Previous studies mainly focused on designing the models solely based on individual data sets and might suffer from degradable performance. Therefore, developing a robust model for a combined data set with diverse knowledge becomes crucial. However, designing the model with a combined data set requires extensive training time and sequential workload to obtain optimal performance without having some prior knowledge about the model's parameters. The presented study here will help solve these issues by introducing the unified training strategy to have a base structure for the classifier and all hyperparameters from individual models using a pretrained transformer model. The performance of the proposed model is noted using three publicly available data sets, namely ISOT and others from the Kaggle website. The results indicate that the proposed unified training strategy surpassed the existing models such as Random Forests, convolutional neural networks, and long short-term memory, with 97% accuracy and achieved the F1 score of 0.97. Furthermore, there was a significant reduction in training time by almost 1.5 to 1.8 × by removing words lower than three letters from the input samples. We also did extensive performance analysis by varying the number of encoder blocks to build compact models and trained on the combined data set. We justify that reducing encoder blocks resulted in lower performance from the obtained results.

Clinical Artificial Intelligence: Design Principles and Fallacies

Clinical artificial intelligence (AI)/machine learning (ML) is anticipated to offer new abilities in clinical decision support, diagnostic reasoning, precision medicine, clinical operational support, and clinical research, but careful concern is needed to ensure these technologies work effectively in the clinic. Here, we detail the clinical ML/AI design process, identifying several key questions and detailing several common forms of issues that arise with ML tools, as motivated by real-world examples, such that clinicians and researchers can better anticipate and correct for such issues in their own use of ML/AI techniques.

Natural language processing for clinical notes in dentistry: A systematic review

OBJECTIVE

To identify and synthesise research on applications of natural language processing (NLP) for information extraction and retrieval from clinical notes in dentistry.

MATERIALS AND METHODS

A predefined search strategy was applied in EMBASE, CINAHL and Medline. Studies eligible for inclusion were those that that described, evaluated, or applied NLP to clinical notes containing either human or simulated patient information. Quality of the study design and reporting was independently assessed based on a set of questions derived from relevant tools including CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS). A narrative synthesis was conducted to present the results.

RESULTS

Of the 17 included studies, 10 developed and evaluated NLP methods and 7 described applications of NLP-based information retrieval methods in dental records. Studies were published between 2015 and 2021, most were missing key details needed for reproducibility, and there was no consistency in design or reporting. The 10 studies developing or evaluating NLP methods used document classification or entity extraction, and 4 compared NLP methods to non-NLP methods. The quality of reporting on NLP studies in dentistry has modestly improved over time.

CONCLUSIONS

Study design heterogeneity and incomplete reporting of studies currently limits our ability to synthesise NLP applications in dental records. Standardisation of reporting and improved connections between NLP methods and applied NLP in dentistry may improve how we can make use of clinical notes from dentistry in population health or decision support systems.

PROTOCOL REGISTRATION

PROSPERO CRD42021227823.

Augmenting DSM-5 diagnostic criteria with self-attention-based BiLSTM models for psychiatric diagnosis

BACKGROUND

Most previous studies make psychiatric diagnoses based on diagnostic terms. In this study we sought to augment Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) diagnostic criteria with deep neural network models to make psychiatric diagnoses based on psychiatric notes.

METHODS

We augmented DSM-5 diagnostic criteria with self-attention-based bidirectional long short-term memory (BiLSTM) models to identify schizophrenia, bipolar, and unipolar depressive disorders. Given that the diagnostic criteria for psychiatric diagnosis include a certain symptom profile and functional impairment, we first extracted psychiatric symptoms and functional features with two approaches, including a lexicon-based approach and a dependency parsing approach. Then, we incorporated free-text discharge notes and extracted features for psychiatric diagnoses with the proposed models.

RESULTS

The micro-averaged F1 scores of the two automatic annotation approaches were greater than 0.8. BiLSTM models with self-attention outperformed the rule-based models with DSM-5 criteria in the prediction of schizophrenia and bipolar disorder, while the latter outperformed the former in predicting unipolar depressive disorder. Approaches for augmenting DSM-5 criteria with a self-attention-based BiLSTM outperformed both pure rule-based and pure deep neural network models. In terms of classification of psychiatric diagnoses, we observed that the performance for schizophrenia and bipolar disorder was acceptable.

CONCLUSION

This DSM-5-augmented deep neural network models showed good performance in identifying psychiatric diagnoses from psychiatric notes. We conclude that it is possible to establish a model that consults clinical notes to make psychiatric diagnoses comparably to physicians. Further research will be extended to outpatient notes and other psychiatric disorders.

Machine learning applied to electronic health record data in home healthcare: A scoping review

OBJECTIVE

Despite recent calls for home healthcare (HHC) to integrate informatics, the application of machine learning in HHC is relatively unknown. Thus, this study aimed to synthesize and appraise the literature describing the application of machine learning to predict adverse outcomes (e.g., hospitalization, mortality) using electronic health record (EHR) data in the HHC setting. Our secondary aim was to evaluate the comprehensiveness of predictors used in the machine learning algorithms guided by the Biopsychosocial Model.

METHODS

During March 2022 we conducted a literature search in four databases: PubMed, Embase, CINAHL, and Scopus. Inclusion criteria were 1) describing services provided in the HHC setting, 2) applying machine learning algorithms to predict adverse outcomes, defined as outcomes related to patient deterioration, 3) using EHR data and 4) focusing on the adult population. Predictors were mapped to the Biopsychosocial Model. A risk of bias analysis was conducted using the Prediction Model Risk Of Bias Assessment Tool.

RESULTS

The final sample included 20 studies. Eighteen studies used predictors from standardized assessments integrated in the EHR. The most common outcome of interest was hospitalization (55%), followed by mortality (25%). Psychological predictors were frequently excluded (35%). Tree based algorithms were most frequently applied (75%). Most studies demonstrated high or unclear risk of bias (75%).

CONCLUSION

Future studies in HHC should consider incorporating machine learning algorithms into clinical decision support systems to identify patients at risk. Based on the Biopsychosocial model, psychological and interpersonal characteristics should be used along with biological characteristics to enhance risk prediction. To facilitate the widespread adoption of machine learning, stakeholders should encourage standardization in the HHC setting.