Identifying Suicide Ideation and Suicidal Attempts in a Psychiatric Clinical Research Database using Natural Language Processing

Sociodemographic and clinical risk factors for suicidal ideation and suicide attempt in functional/dissociative seizures and epilepsy: a large cohort study

BACKGROUND

People with functional/dissociative seizures (FDS) are at elevated suicidality risk.

OBJECTIVE

To identify risk factors for suicidality in FDS or epilepsy.

METHODS

Retrospective cohort study from the UK's largest tertiary mental healthcare provider, with linked national admission data from the Hospital Episode Statistics. Participants were 2383 people with a primary or secondary diagnosis of FDS or epilepsy attending between 01 January 2007 and 18 June 2021. Outcomes were a first report of suicidal ideation and a first hospital admission for suicide attempt (International Classification of Diseases, version 10: X60-X84). Demographic and clinical risk factors were assessed using multivariable bias-reduced binomial-response generalised linear models.

FINDINGS

In both groups, ethnic minorities had significantly reduced odds of hospitalisation following suicide attempt (OR: 0.45-0.49). Disorder-specific risk factors were gender, age and comorbidity profile. In FDS, both genders had similar suicidality risk; younger age was a risk factor for both outcomes (OR: 0.16-1.91). A diagnosis of depression or personality disorders was associated with higher odds of suicidal ideation (OR: 1.91-3.01). In epilepsy, females had higher odds of suicide attempt-related hospitalisation (OR: 1.64). Age had a quadratic association with both outcomes (OR: 0.88-1.06). A substance abuse disorder was associated with higher suicidal ideation (OR: 2.67). Developmental disorders lowered the risk (OR: 0.16-0.24).

CONCLUSIONS

This is the first study systematically reporting risk factors for suicidality in people with FDS. Results for the large epilepsy cohort complement previous studies and will be useful in future meta-analyses.

CLINICAL IMPLICATIONS

Risk factors identified will help identify higher-risk groups in clinical settings.

Predicting which patients with cancer will see a psychiatrist or counsellor from their initial oncology consultation document using natural language processing

BACKGROUND

Patients with cancer often have unmet psychosocial needs. Early detection of who requires referral to a counsellor or psychiatrist may improve their care. This work used natural language processing to predict which patients will see a counsellor or psychiatrist from a patient's initial oncology consultation document. We believe this is the first use of artificial intelligence to predict psychiatric outcomes from non-psychiatric medical documents.

METHODS

This retrospective prognostic study used data from 47,625 patients at BC Cancer. We analyzed initial oncology consultation documents using traditional and neural language models to predict whether patients would see a counsellor or psychiatrist in the 12 months following their initial oncology consultation.

RESULTS

Here, we show our best models achieved a balanced accuracy (receiver-operating-characteristic area-under-curve) of 73.1% (0.824) for predicting seeing a psychiatrist, and 71.0% (0.784) for seeing a counsellor. Different words and phrases are important for predicting each outcome.

CONCLUSION

These results suggest natural language processing can be used to predict psychosocial needs of patients with cancer from their initial oncology consultation document. Future research could extend this work to predict the psychosocial needs of medical patients in other settings.

Detection of Personal and Family History of Suicidal Thoughts and Behaviors using Deep Learning and Natural Language Processing: A Multi-Site Study

Objective

Personal and family history of suicidal thoughts and behaviors (PSH and FSH, respectively) are significant risk factors associated with future suicide events. These are often captured in narrative clinical notes in electronic health records (EHRs). Collaboratively, Weill Cornell Medicine (WCM), Northwestern Medicine (NM), and the University of Florida (UF) developed and validated deep learning (DL)-based natural language processing (NLP) tools to detect PSH and FSH from such notes. The tool's performance was further benchmarked against a method relying exclusively on ICD-9/10 diagnosis codes.

Materials and Methods

We developed DL-based NLP tools utilizing pre-trained transformer models Bio_ClinicalBERT and GatorTron, and compared them with expert-informed, rule-based methods. The tools were initially developed and validated using manually annotated clinical notes at WCM. Their portability and performance were further evaluated using clinical notes at NM and UF.

Results

The DL tools outperformed the rule-based NLP tool in identifying PSH and FHS. For detecting PSH, the rule-based system obtained an F1-score of 0.75 ± 0.07, while the Bio_ClinicalBERT and GatorTron DL tools scored 0.83 ± 0.09 and 0.84 ± 0.07, respectively. For detecting FSH, the rule-based NLP tool's F1-score was 0.69 ± 0.11, compared to 0.89 ± 0.10 for Bio_ClinicalBERT and 0.92 ± 0.07 for GatorTron. For the gold standard corpora across the three sites, only 2.2% (WCM), 9.3% (NM), and 7.8% (UF) of patients reported to have an ICD-9/10 diagnosis code for suicidal thoughts and behaviors prior to the clinical notes report date. The best performing GatorTron DL tool identified 93.0% (WCM), 80.4% (NM), and 89.0% (UF) of patients with documented PSH, and 85.0%(WCM), 89.5%(NM), and 100%(UF) of patients with documented FSH in their notes.

Discussion

While PSH and FSH are significant risk factors for future suicide events, little effort has been made previously to identify individuals with these history. To address this, we developed a transformer based DL method and compared with conventional rule-based NLP approach. The varying effectiveness of the rule-based tools across sites suggests a need for improvement in its dictionary-based approach. In contrast, the performances of the DL tools were higher and comparable across sites. Furthermore, DL tools were fine-tuned using only small number of annotated notes at each site, underscores its greater adaptability to local documentation practices and lexical variations.

Conclusion

Variations in local documentation practices across health care systems pose challenges to rule-based NLP tools. In contrast, the developed DL tools can effectively extract PSH and FSH information from unstructured clinical notes. These tools will provide clinicians with crucial information for assessing and treating patients at elevated risk for suicide who are rarely been diagnosed.

Machine learning and the prediction of suicide in psychiatric populations: a systematic review

Machine learning (ML) has emerged as a promising tool to enhance suicidal prediction. However, as many large-sample studies mixed psychiatric and non-psychiatric populations, a formal psychiatric diagnosis emerged as a strong predictor of suicidal risk, overshadowing more subtle risk factors specific to distinct populations. To overcome this limitation, we conducted a systematic review of ML studies evaluating suicidal behaviors exclusively in psychiatric clinical populations. A systematic literature search was performed from inception through November 17, 2022 on PubMed, EMBASE, and Scopus following the PRISMA guidelines. Original research using ML techniques to assess the risk of suicide or predict suicide attempts in the psychiatric population were included. An assessment for bias risk was performed using the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) guidelines. About 1032 studies were retrieved, and 81 satisfied the inclusion criteria and were included for qualitative synthesis. Clinical and demographic features were the most frequently employed and random forest, support vector machine, and convolutional neural network performed better in terms of accuracy than other algorithms when directly compared. Despite heterogeneity in procedures, most studies reported an accuracy of 70% or greater based on features such as previous attempts, severity of the disorder, and pharmacological treatments. Although the evidence reported is promising, ML algorithms for suicidal prediction still present limitations, including the lack of neurobiological and imaging data and the lack of external validation samples. Overcoming these issues may lead to the development of models to adopt in clinical practice. Further research is warranted to boost a field that holds the potential to critically impact suicide mortality.

Suicide-related behavior and firearm access among perpetrators of domestic violence subject to domestic violence protection orders

Perpetrators of domestic violence (DV) may be a population at elevated risk of suicide. Domestic violence protection orders (DVPOs) can include the removal of firearms from the individual subjected to the order (i.e., the respondent) to protect the victim-survivor. While removal of firearms in a DVPO is designed to protect the victim-survivor; it may also prevent suicide of the respondent by reducing access to lethal means. Therefore, we examined the association of respondent suicide-related behaviors with firearm possession and weapon use in DV among a sample of granted DVPO petitions in King County, Washington (WA), United States from 2014 to 2020 (n = 2,537). We compared prevalence ratios (PR) of respondent firearm possession and use of firearms or weapons to threaten or harm by suicide-related behavior. Overall, respondent suicide-related behavior was commonly reported by petitioners (46 %). Approximately 30 % of respondents possessed firearms. This was similar between respondents with and without a history of suicide-related behavior (PR: 1.03; 95 % CI: 0.91-1.17). Respondents with a history of suicide-related behavior were 1.33 times more likely to have used firearms or weapons to threaten/harm in DV compared to those without a history of suicide-related behavior (44.1 % vs. 33.8 %; 95 % CI: 1.20-1.47). In conclusion, both firearm possession and suicide-related behaviors were common among DVPO respondents. History of suicide-related behavior may be a marker for firearm-related harm to the victim-survivor. Evaluations of DVPO firearm dispossession should consider both firearm-related injury of the victim-survivor and suicide of the respondent.

Combining clinical notes with structured electronic health records enhances the prediction of mental health crises

An automatic prediction of mental health crises can improve caseload prioritization and enable preventative interventions, improving patient outcomes and reducing costs. We combine structured electronic health records (EHRs) with clinical notes from 59,750 de-identified patients to predict the risk of mental health crisis relapse within the next 28 days. The results suggest that an ensemble machine learning model that relies on structured EHRs and clinical notes when available, and relying solely on structured data when the notes are unavailable, offers superior performance over models trained with either of the two data streams alone. Furthermore, the study provides key takeaways related to the required amount of clinical notes to add value in predictive analytics. This study sheds light on the untapped potential of clinical notes in the prediction of mental health crises and highlights the importance of choosing an appropriate machine learning method to combine structured and unstructured EHRs.

Suicide Risk Assessments Through the Eyes of ChatGPT-3.5 Versus ChatGPT-4: Vignette Study

BACKGROUND

ChatGPT, a linguistic artificial intelligence (AI) model engineered by OpenAI, offers prospective contributions to mental health professionals. Although having significant theoretical implications, ChatGPT's practical capabilities, particularly regarding suicide prevention, have not yet been substantiated.

OBJECTIVE

The study's aim was to evaluate ChatGPT's ability to assess suicide risk, taking into consideration 2 discernable factors-perceived burdensomeness and thwarted belongingness-over a 2-month period. In addition, we evaluated whether ChatGPT-4 more accurately evaluated suicide risk than did ChatGPT-3.5.

METHODS

ChatGPT was tasked with assessing a vignette that depicted a hypothetical patient exhibiting differing degrees of perceived burdensomeness and thwarted belongingness. The assessments generated by ChatGPT were subsequently contrasted with standard evaluations rendered by mental health professionals. Using both ChatGPT-3.5 and ChatGPT-4 (May 24, 2023), we executed 3 evaluative procedures in June and July 2023. Our intent was to scrutinize ChatGPT-4's proficiency in assessing various facets of suicide risk in relation to the evaluative abilities of both mental health professionals and an earlier version of ChatGPT-3.5 (March 14 version).

RESULTS

During the period of June and July 2023, we found that the likelihood of suicide attempts as evaluated by ChatGPT-4 was similar to the norms of mental health professionals (n=379) under all conditions (average Z score of 0.01). Nonetheless, a pronounced discrepancy was observed regarding the assessments performed by ChatGPT-3.5 (May version), which markedly underestimated the potential for suicide attempts, in comparison to the assessments carried out by the mental health professionals (average Z score of -0.83). The empirical evidence suggests that ChatGPT-4's evaluation of the incidence of suicidal ideation and psychache was higher than that of the mental health professionals (average Z score of 0.47 and 1.00, respectively). Conversely, the level of resilience as assessed by both ChatGPT-4 and ChatGPT-3.5 (both versions) was observed to be lower in comparison to the assessments offered by mental health professionals (average Z score of -0.89 and -0.90, respectively).

CONCLUSIONS

The findings suggest that ChatGPT-4 estimates the likelihood of suicide attempts in a manner akin to evaluations provided by professionals. In terms of recognizing suicidal ideation, ChatGPT-4 appears to be more precise. However, regarding psychache, there was an observed overestimation by ChatGPT-4, indicating a need for further research. These results have implications regarding ChatGPT-4's potential to support gatekeepers, patients, and even mental health professionals' decision-making. Despite the clinical potential, intensive follow-up studies are necessary to establish the use of ChatGPT-4's capabilities in clinical practice. The finding that ChatGPT-3.5 frequently underestimates suicide risk, especially in severe cases, is particularly troubling. It indicates that ChatGPT may downplay one's actual suicide risk level.

Identifying suicide documentation in clinical notes through zero-shot learning

Background and Aims

In deep learning, a major difficulty in identifying suicidality and its risk factors in clinical notes is the lack of training samples given the small number of true positive instances among the number of patients screened. This paper describes a novel methodology that identifies suicidality in clinical notes by addressing this data sparsity issue through zero-shot learning. Our general aim was to develop a tool that leveraged zero-shot learning to effectively identify suicidality documentation in all types of clinical notes.

Methods

US Veterans Affairs clinical notes served as data. The training data set label was determined using diagnostic codes of suicide attempt and self-harm. We used a base string associated with the target label of suicidality to provide auxiliary information by narrowing the positive training cases to those containing the base string. We trained a deep neural network by mapping the training documents' contents to a semantic space. For comparison, we trained another deep neural network using the identical training data set labels, and bag-of-words features.

Results

The zero-shot learning model outperformed the baseline model in terms of area under the curve, sensitivity, specificity, and positive predictive value at multiple probability thresholds. In applying a 0.90 probability threshold, the methodology identified notes documenting suicidality but not associated with a relevant ICD-10-CM code, with 94% accuracy.

Conclusion

This method can effectively identify suicidality without manual annotation.

A Call to Action on Assessing and Mitigating Bias in Artificial Intelligence Applications for Mental Health

Advances in computer science and data-analytic methods are driving a new era in mental health research and application. Artificial intelligence (AI) technologies hold the potential to enhance the assessment, diagnosis, and treatment of people experiencing mental health problems and to increase the reach and impact of mental health care. However, AI applications will not mitigate mental health disparities if they are built from historical data that reflect underlying social biases and inequities. AI models biased against sensitive classes could reinforce and even perpetuate existing inequities if these models create legacies that differentially impact who is diagnosed and treated, and how effectively. The current article reviews the health-equity implications of applying AI to mental health problems, outlines state-of-the-art methods for assessing and mitigating algorithmic bias, and presents a call to action to guide the development of fair-aware AI in psychological science.

Enriching Real-world Data with Social Determinants of Health for Health Outcomes and Health Equity: Successes, Challenges, and Opportunities

OBJECTIVE

To summarize the recent methods and applications that leverage real-world data such as electronic health records (EHRs) with social determinants of health (SDoH) for public and population health and health equity and identify successes, challenges, and possible solutions.

METHODS

In this opinion review, grounded on a social-ecological-model-based conceptual framework, we surveyed data sources and recent informatics approaches that enable leveraging SDoH along with real-world data to support public health and clinical health applications including helping design public health intervention, enhancing risk stratification, and enabling the prediction of unmet social needs.

RESULTS

Besides summarizing data sources, we identified gaps in capturing SDoH data in existing EHR systems and opportunities to leverage informatics approaches to collect SDoH information either from structured and unstructured EHR data or through linking with public surveys and environmental data. We also surveyed recently developed ontologies for standardizing SDoH information and approaches that incorporate SDoH for disease risk stratification, public health crisis prediction, and development of tailored interventions.

CONCLUSIONS

To enable effective public health and clinical applications using real-world data with SDoH, it is necessary to develop both non-technical solutions involving incentives, policies, and training as well as technical solutions such as novel social risk management tools that are integrated into clinical workflow. Ultimately, SDoH-powered social risk management, disease risk prediction, and development of SDoH tailored interventions for disease prevention and management have the potential to improve population health, reduce disparities, and improve health equity.

A multisite comparison using electronic health records and natural language processing to identify the association between suicidality and hospital readmission amongst patients with eating disorders

OBJECTIVES

To describe and compare the association between suicidality and subsequent readmission for patients hospitalized for eating disorder treatment, within 2 years of discharge, at two large academic medical centers in two different countries.

METHODS

Over an 8-year study window from January 2009 to March 2017, we identified all inpatient eating disorder admissions at Weill Cornell Medicine, New York, USA (WCM) and South London and Maudsley Foundation NHS Trust, London, UK (SLaM). To establish each patient's-suicidality profile, we applied two natural language processing (NLP) algorithms, independently developed at the two institutions, and detected suicidality in clinical notes documented in the first week of admission. We calculated the odds ratios (OR) for any subsequent readmission within 2 years postdischarge and determined whether this was to another eating disorder unit, other psychiatric unit, a general medical hospital admission or emergency room attendance.

RESULTS

We identified 1126 and 420 eating disorder inpatient admissions at WCM and SLaM, respectively. In the WCM cohort, evidence of above average suicidality during the first week of admission was significantly associated with an increased risk of noneating disorder-related psychiatric readmission (OR 3.48 95% CI = 2.03-5.99, p-value < .001), but a similar pattern was not observed in the SLaM cohort (OR 1.34, 95% CI = 0.75-2.37, p = .32), there was no significant increase in risk of admission. In both cohorts, personality disorder increased the risk of any psychiatric readmission within 2 years.

DISCUSSION

Patterns of increased risk of psychiatric readmission from above average suicidality detected via NLP during inpatient eating disorder admissions differed in our two patient cohorts. However, comorbid diagnoses such as personality disorder increased the risk of any psychiatric readmission across both cohorts.

PUBLIC SIGNIFICANCE

Suicidality amongst is eating disorders is an extremely common presentation and it is important we further our understanding of identifying those most at risk. This research also provides a novel study design, comparing two NLP algorithms on electronic health record data based in the United States and United Kingdom on eating disorder inpatients. Studies researching both UK and US mental health patients are sparse therefore this study provides novel data.

A Natural Language Processing Algorithm for Classifying Suicidal Behaviors in Alzheimer's Disease and Related Dementia Patients: Development and Validation Using Electronic Health Records Data

This study aimed to develop a natural language processing algorithm (NLP) using machine learning (ML) and Deep Learning (DL) techniques to identify and classify documentation of suicidal behaviors in patients with Alzheimer's disease and related dementia (ADRD). We utilized MIMIC-III and MIMIC-IV datasets and identified ADRD patients and subsequently those with suicide ideation using relevant International Classification of Diseases (ICD) codes. We used cosine similarity with ScAN (Suicide Attempt and Ideation Events Dataset) to calculate semantic similarity scores of ScAN with extracted notes from MIMIC for the clinical notes. The notes were sorted based on these scores, and manual review and categorization into eight suicidal behavior categories were performed. The data were further analyzed using conventional ML and DL models, with manual annotation as a reference. The tested classifiers achieved classification results close to human performance with up to 98% precision and 98% recall of suicidal ideation in the ADRD patient population. Our NLP model effectively reproduced human annotation of suicidal ideation within the MIMIC dataset. These results establish a foundation for identifying and categorizing documentation related to suicidal ideation within ADRD population, contributing to the advancement of NLP techniques in healthcare for extracting and classifying clinical concepts, particularly focusing on suicidal ideation among patients with ADRD. Our study showcased the capability of a robust NLP algorithm to accurately identify and classify documentation of suicidal behaviors in ADRD patients.

Dynamic suicide topic modelling: Deriving population-specific, psychosocial and time-sensitive suicide risk variables from Electronic Health Record psychotherapy notes

In the machine learning subfield of natural language processing, a topic model is a type of unsupervised method that is used to uncover abstract topics within a corpus of text. Dynamic topic modelling (DTM) is used for capturing change in these topics over time. The study deploys DTM on corpus of electronic health record psychotherapy notes. This retrospective study examines whether DTM helps distinguish closely matched patients that did and did not die by suicide. Cohort consists of United States Department of Veterans Affairs (VA) patients diagnosed with Posttraumatic Stress Disorder (PTSD) between 2004 and 2013. Each case (those who died by suicide during the year following diagnosis) was matched with five controls (those who remained alive) that shared psychotherapists and had similar suicide risk based on VA's suicide prediction algorithm. Cohort was restricted to patients who received psychotherapy for 9+ months after initial PTSD diagnoses (cases = 77; controls = 362). For cases, psychotherapy notes from diagnosis until death were examined. For controls, psychotherapy notes from diagnosis until matched case's death date were examined. A Python-based DTM algorithm was utilized. Derived topics identified population-specific themes, including PTSD, psychotherapy, medication, communication and relationships. Control topics changed significantly more over time than case topics. Topic differences highlighted engagement, expressivity and therapeutic alliance. This study strengthens groundwork for deriving population-specific, psychosocial and time-sensitive suicide risk variables.

Development of a Corpus Annotated With Mentions of Pain in Mental Health Records: Natural Language Processing Approach

BACKGROUND

Pain is a widespread issue, with 20% of adults (1 in 5) experiencing it globally. A strong association has been demonstrated between pain and mental health conditions, and this association is known to exacerbate disability and impairment. Pain is also known to be strongly related to emotions, which can lead to damaging consequences. As pain is a common reason for people to access health care facilities, electronic health records (EHRs) are a potential source of information on this pain. Mental health EHRs could be particularly beneficial since they can show the overlap of pain with mental health. Most mental health EHRs contain the majority of their information within the free-text sections of the records. However, it is challenging to extract information from free text. Natural language processing (NLP) methods are therefore required to extract this information from the text.

OBJECTIVE

This research describes the development of a corpus of manually labeled mentions of pain and pain-related entities from the documents of a mental health EHR database, for use in the development and evaluation of future NLP methods.

METHODS

The EHR database used, Clinical Record Interactive Search, consists of anonymized patient records from The South London and Maudsley National Health Service Foundation Trust in the United Kingdom. The corpus was developed through a process of manual annotation where pain mentions were marked as relevant (ie, referring to physical pain afflicting the patient), negated (ie, indicating absence of pain), or not relevant (ie, referring to pain affecting someone other than the patient, or metaphorical and hypothetical mentions). Relevant mentions were also annotated with additional attributes such as anatomical location affected by pain, pain character, and pain management measures, if mentioned.

RESULTS

A total of 5644 annotations were collected from 1985 documents (723 patients). Over 70% (n=4028) of the mentions found within the documents were annotated as relevant, and about half of these mentions also included the anatomical location affected by the pain. The most common pain character was chronic pain, and the most commonly mentioned anatomical location was the chest. Most annotations (n=1857, 33%) were from patients who had a primary diagnosis of mood disorders (International Classification of Diseases-10th edition, chapter F30-39).

CONCLUSIONS

This research has helped better understand how pain is mentioned within the context of mental health EHRs and provided insight into the kind of information that is typically mentioned around pain in such a data source. In future work, the extracted information will be used to develop and evaluate a machine learning-based NLP application to automatically extract relevant pain information from EHR databases.

The use of advanced technology and statistical methods to predict and prevent suicide

In the past decade, two themes have emerged across suicide research. First, according to meta-analyses, the ability to predict and prevent suicidal thoughts and behaviours is weaker than would be expected for the size of the field. Second, review and commentary papers propose that technological and statistical methods (such as smartphones, wearables, digital phenotyping and machine learning) might become solutions to this problem. In this Review, we aim to strike a balance between the pessimistic picture presented by these meta-analyses and the optimistic picture presented by review and commentary papers about the promise of advanced technological and statistical methods to improve the ability to understand, predict and prevent suicide. We divide our discussion into two broad categories. First, we discuss the research aimed at assessment, with the goal of better understanding or more accurately predicting suicidal thoughts and behaviours. Second, we discuss the literature that focuses on prevention of suicidal thoughts and behaviours. Ecological momentary assessment, wearables and other technological and statistical advances hold great promise for predicting and preventing suicide, but there is much yet to do.

Investigating online activity in UK adolescent mental health patients: a feasibility study using a natural language processing approach for electronic health records

OBJECTIVES

To assess the feasibility of using a natural language processing (NLP) application for extraction of free-text online activity mentions in adolescent mental health patient electronic health records (EHRs).

SETTING

The Clinical Records Interactive Search system allows detailed research based on deidentified EHRs from the South London and Maudsley NHS Foundation Trust, a large south London Mental Health Trust providing secondary and tertiary mental healthcare.

PARTICIPANTS AND METHODS

We developed a gazetteer of online activity terms and annotation guidelines, from 5480 clinical notes (200 adolescents, aged 11-17 years) receiving specialist mental healthcare. The preprocessing and manual curation steps of this real-world data set allowed development of a rule-based NLP application to automate identification of online activity (internet, social media, online gaming) mentions in EHRs. The context of each mention was also recorded manually as: supportive, detrimental or neutral in a subset of data for additional analysis.

RESULTS

The NLP application performed with good precision (0.97) and recall (0.94) for identification of online activity mentions. Preliminary analyses found 34% of online activity mentions were considered to have been documented within a supportive context for the young person, 38% detrimental and 28% neutral.

CONCLUSION

Our results provide an important example of a rule-based NLP methodology to accurately identify online activity recording in EHRs, enabling researchers to now investigate associations with a range of adolescent mental health outcomes.

A machine learning approach to identifying suicide risk among text-based crisis counseling encounters

Introduction

With the increasing utilization of text-based suicide crisis counseling, new means of identifying at risk clients must be explored. Natural language processing (NLP) holds promise for evaluating the content of crisis counseling; here we use a data-driven approach to evaluate NLP methods in identifying client suicide risk.

Methods

De-identified crisis counseling data from a regional text-based crisis encounter and mobile tipline application were used to evaluate two modeling approaches in classifying client suicide risk levels. A manual evaluation of model errors and system behavior was conducted.

Results

The neural model outperformed a term frequency-inverse document frequency (tf-idf) model in the false-negative rate. While 75% of the neural model's false negative encounters had some discussion of suicidality, 62.5% saw a resolution of the client's initial concerns. Similarly, the neural model detected signals of suicidality in 60.6% of false-positive encounters.

Discussion

The neural model demonstrated greater sensitivity in the detection of client suicide risk. A manual assessment of errors and model performance reflected these same findings, detecting higher levels of risk in many of the false-positive encounters and lower levels of risk in many of the false negatives. NLP-based models can detect the suicide risk of text-based crisis encounters from the encounter's content.

Application of Natural Language Processing (NLP) in Detecting and Preventing Suicide Ideation: A Systematic Review

(1) Introduction: Around a million people are reported to die by suicide every year, and due to the stigma associated with the nature of the death, this figure is usually assumed to be an underestimate. Machine learning and artificial intelligence such as natural language processing has the potential to become a major technique for the detection, diagnosis, and treatment of people. (2) Methods: PubMed, EMBASE, MEDLINE, PsycInfo, and Global Health databases were searched for studies that reported use of NLP for suicide ideation or self-harm. (3) Result: The preliminary search of 5 databases generated 387 results. Removal of duplicates resulted in 158 potentially suitable studies. Twenty papers were finally included in this review. (4) Discussion: Studies show that combining structured and unstructured data in NLP data modelling yielded more accurate results than utilizing either alone. Additionally, to reduce suicides, people with mental problems must be continuously and passively monitored. (5) Conclusions: The use of AI&ML opens new avenues for considerably guiding risk prediction and advancing suicide prevention frameworks. The review's analysis of the included research revealed that the use of NLP may result in low-cost and effective alternatives to existing resource-intensive methods of suicide prevention.

Modifiable risk factors for inpatient violence in psychiatric hospital: prospective study and prediction model

BACKGROUND

Violence perpetrated by psychiatric inpatients is associated with modifiable factors. Current structured approaches to assess inpatient violence risk lack predictive validity and linkage to interventions.

METHODS

Adult psychiatric inpatients on forensic and general wards in three psychiatric hospitals were recruited and followed up prospectively for 6 months. Information on modifiable (dynamic) risk factors were collected every 1-4 weeks, and baseline background factors. Data were transferred to a web-based monitoring system (FOxWeb) to calculate a total dynamic risk score. Outcomes were extracted from an incident-reporting system recording aggression and interpersonal violence. The association between total dynamic score and violent incidents was assessed by multilevel logistic regression and compared with dynamic score excluded.

RESULTS

We recruited 89 patients and conducted 624 separate assessments (median 5/patient). Mean age was 39 (s.d. 12.5) years with 20% (n = 18) female. Common diagnoses were schizophrenia-spectrum disorders (70%, n = 62) and personality disorders (20%, n = 18). There were 93 violent incidents. Factors contributing to violence risk were a total dynamic score of ⩾1 (OR 3.39, 95% CI 1.25-9.20), 10-year increase in age (OR 0.67, 0.47-0.96), and female sex (OR 2.78, 1.04-7.40). Non-significant associations with schizophrenia-spectrum disorder were found (OR 0.50, 0.20-1.21). In a fixed-effect model using all covariates, AUC was 0.77 (0.72-0.82) and 0.75 (0.70-0.80) when the dynamic score was excluded.

CONCLUSIONS

In predicting violence risk in individuals with psychiatric disorders, modifiable factors added little incremental value beyond static ones in a psychiatric inpatient setting. Future work should make a clear distinction between risk factors that assist in prediction and those linked to needs.

Natural language processing and network analysis in patients withdrawing from life-sustaining treatments: a retrospective cohort study

BACKGROUND

Providing palliative care to patients who withdraw from life-sustaining treatments is crucial; however, delays or the absence of such services are prevalent. This study used natural language processing and network analysis to identify the role of medications as early palliative care referral triggers.

METHODS

We conducted a retrospective observational study of 119 adult patients receiving specialized palliative care after endotracheal tube withdrawal in intensive care units of a Taiwan-based medical center between July 2016 and June 2018. Patients were categorized into early integration and late referral groups based on the median survival time. Using natural language processing, we analyzed free texts from electronic health records. The Palliative trigger index was also calculated for comparison, and network analysis was performed to determine the co-occurrence of terms between the two groups.

RESULTS

Broad-spectrum antibiotics, antifungal agents, diuretics, and opioids had high Palliative trigger index. The most common co-occurrences in the early integration group were micafungin and voriconazole (co-correlation = 0.75). However, in the late referral group, piperacillin and penicillin were the most common co-occurrences (co-correlation = 0.843).

CONCLUSION

Treatments for severe infections, chronic illnesses, and analgesics are possible triggers for specialized palliative care consultations. The Palliative trigger index and network analysis indicated the need for palliative care in patients withdrawing from life-sustaining treatments. This study recommends establishing a therapeutic control system based on computerized order entry and integrating it into a shared-decision model.

Portability of natural language processing methods to detect suicidality from clinical text in US and UK electronic health records

Background

In the global effort to prevent death by suicide, many academic medical institutions are implementing natural language processing (NLP) approaches to detect suicidality from unstructured clinical text in electronic health records (EHRs), with the hope of targeting timely, preventative interventions to individuals most at risk of suicide. Despite the international need, the development of these NLP approaches in EHRs has been largely local and not shared across healthcare systems.

Methods

In this study, we developed a process to share NLP approaches that were individually developed at King's College London (KCL), UK and Weill Cornell Medicine (WCM), US - two academic medical centers based in different countries with vastly different healthcare systems. We tested and compared the algorithms' performance on manually annotated clinical notes (KCL: n = 4,911 and WCM = 837).

Results

After a successful technical porting of the NLP approaches, our quantitative evaluation determined that independently developed NLP approaches can detect suicidality at another healthcare organization with a different EHR system, clinical documentation processes, and culture, yet do not achieve the same level of success as at the institution where the NLP algorithm was developed (KCL approach: F1-score 0.85 vs. 0.68, WCM approach: F1-score 0.87 vs. 0.72).

Limitations

Independent NLP algorithm development and patient cohort selection at the two institutions comprised direct comparability.

Conclusions

Shared use of these NLP approaches is a critical step forward towards improving data-driven algorithms for early suicide risk identification and timely prevention.

Improving ascertainment of suicidal ideation and suicide attempt with natural language processing

Methods relying on diagnostic codes to identify suicidal ideation and suicide attempt in Electronic Health Records (EHRs) at scale are suboptimal because suicide-related outcomes are heavily under-coded. We propose to improve the ascertainment of suicidal outcomes using natural language processing (NLP). We developed information retrieval methodologies to search over 200 million notes from the Vanderbilt EHR. Suicide query terms were extracted using word2vec. A weakly supervised approach was designed to label cases of suicidal outcomes. The NLP validation of the top 200 retrieved patients showed high performance for suicidal ideation (area under the receiver operator curve [AUROC]: 98.6, 95% confidence interval [CI] 97.1–99.5) and suicide attempt (AUROC: 97.3, 95% CI 95.2–98.7). Case extraction produced the best performance when combining NLP and diagnostic codes and when accounting for negated suicide expressions in notes. Overall, we demonstrated that scalable and accurate NLP methods can be developed to identify suicidal behavior in EHRs to enhance prevention efforts, predictive models, and precision medicine.

Leveraging unstructured electronic medical record notes to derive population-specific suicide risk models

Electronic medical record (EMR)-based suicide risk prediction methods typically rely on analysis of structured variables such as demographics, visit history, and prescription data. Leveraging unstructured EMR notes may improve predictive accuracy by allowing access to nuanced clinical information. We utilized natural language processing (NLP) to analyze a large EMR note corpus to develop a data-driven suicide risk prediction model. We developed a matched case-control sample of U.S. Department of Veterans Affairs (VA) patients in 2015 and 2016. We randomly matched each case (all patients that died by suicide in that interval, n = 5029) with five controls (patients that remained alive). We processed note corpus using NLP methods and applied machine-learning classification algorithms to output. We calculated area under the curve (AUC) and risk tiers to determine predictive accuracy. NLP-derived models demonstrated strong predictive accuracy. Patients that scored within top 10% of risk model accounted for up to 29% of suicide decedents. NLP-derived model compares positively to other leading prediction methods. Our approach is highly implementable, only requiring access to text data and open-source software. Additional studies should evaluate ensemble models incorporating NLP-derived information alongside more typical structured variables.

Critical target identification and human health risk ranking of metal ions based on mechanism-driven modeling

Huge amounts of metals have been released into environment due to various anthropogenic activities, such as smelting and processing of metals and subsequent application in construction, automobiles, batteries, optoelectronic devices, and so on, resulting in widespread detection in environmental media. However, some metal ions are considered as "Environmental health hazards", leading to serious human health concerns through affecting critical targets. Hence, it is necessary to quickly and effectively recognize the key target of metal ions in living organisms. Fortunately, the development of high-throughput analysis and in silico approaches offer a promising tool for target identification. In this study, the key oncogenic target (tumor suppressor protein, p53) was screened by network analysis based on the comparative toxicogenomics database (CTD). Some metal ions could bind to p53 core domain, impair its function and induce the development of cancer risk, but its mechanisms were still unclear. Therefore, a quantitative structure-activity relationship (QSAR) model was constructed to characterize the binding constants (K_a) between DNA binding domain of p53 (p53 DBD) and nine metal ions (Mg²⁺, Ca²⁺, Cu²⁺, Zn²⁺, Co²⁺, Ni²⁺, Mn²⁺, Fe³⁺ and Ba²⁺). It had good robustness and predictive ability, which could be used to predict the K_a values of other six metal ions (Li⁺, Ag⁺, Cs⁺, Cd²⁺, Hg²⁺ and Pb²⁺) within application domain. The results showed strong binding affinity between Cd²⁺/Hg²⁺/Pb²⁺ and p53 DBD. Subsequent mechanism analyses revealed that first hydrolysis constant (|logK_OH|) and polarization force (Z²/r) were key metal ion-characteristic parameters. The metal ions with weak hydrolysis constants and strong polarization forces could readily interact with N-containing histidine and S-containing cysteine of p53 DBD, which resulted in high K_a values. This study identified p53 as potential target for metal ions, revealed the key characteristics affecting the actions and provide a basic understanding of metal ions-p53 DBD interaction.

A Critical Review of Text Mining Applications for Suicide Research

Purpose of Review

Applying text mining to suicide research holds a great deal of promise. In this manuscript, literature from 2019 to 2021 is critically reviewed for text mining projects that use electronic health records, social media data, and death records.

Recent Findings

Text mining has helped identify risk factors for suicide in general and specific populations (e.g., older adults), has been combined with structured variables in EHRs to predict suicide risk, and has been used to track trends in social media suicidal discourse following population level events (e.g., COVID-19, celebrity suicides).

Summary

Future research should utilize text mining along with data linkage methods to capture more complete information on risk factors and outcomes across data sources (e.g., combining death records and EHRs), evaluate effectiveness of NLP-based intervention programs that use suicide risk prediction, establish standards for reporting accuracy of text mining programs to enable comparison across studies, and incorporate implementation science to understand feasibility, acceptability, and technical considerations.

Use of Artificial Intelligence-Based Strategies for Assessing Suicidal Behavior and Mental Illness: A Literature Review

Mental illness leading to suicide attempts is prevalent in a large portion of the population especially in low and middle-income nations. There remains a significant social stigma associated with mental illness that can lead to stigmatization of patients. Hence, patients are reluctant to communicate their problems to health care providers. Physicians have difficulty in timely identification of patients at risk for suicide. Novel and rigorously designed strategies are needed to determine the population at risk for suicide. This would be the first step in overcoming the multitude of barriers in the management of mental illness. Clinical tools and the use of electronic medical records (EMR) are time intensive. Recently, several artificial intelligence (AI)-based predictive technologies have gained momentum. The aim of this review is to summarize the recent advances in this landscape.

ScAN: Suicide Attempt and Ideation Events Dataset

Suicide is an important public health concern and one of the leading causes of death worldwide. Suicidal behaviors, including suicide attempts (SA) and suicide ideations (SI), are leading risk factors for death by suicide. Information related to patients' previous and current SA and SI are frequently documented in the electronic health record (EHR) notes. Accurate detection of such documentation may help improve surveillance and predictions of patients' suicidal behaviors and alert medical professionals for suicide prevention efforts. In this study, we first built Suicide Attempt and Ideation Events (ScAN) dataset, a subset of the publicly available MIMIC III dataset spanning over 12k+ EHR notes with 19k+ annotated SA and SI events information. The annotations also contain attributes such as method of suicide attempt. We also provide a strong baseline model ScANER (Suicide Attempt and Ideation Events Retreiver), a multi-task RoBERTa-based model with a retrieval module to extract all the relevant suicidal behavioral evidences from EHR notes of an hospital-stay and, and a prediction module to identify the type of suicidal behavior (SA and SI) concluded during the patient's stay at the hospital. ScANER achieved a macro-weighted F1-score of 0.83 for identifying suicidal behavioral evidences and a macro F1-score of 0.78 and 0.60 for classification of SA and SI for the patient's hospital-stay, respectively. ScAN and ScANER are publicly available.

Towards Building a Visual Behaviour Analysis Pipeline for Suicide Detection and Prevention

Understanding human behaviours through video analysis has seen significant research progress in recent years with the advancement of deep learning. This topic is of great importance to the next generation of intelligent visual surveillance systems which are capable of real-time detection and analysis of human behaviours. One important application is to automatically monitor and detect individuals who are in crisis at suicide hotspots to facilitate early intervention and prevention. However, there is still a significant gap between research in human action recognition and visual video processing in general, and their application to monitor hotspots for suicide prevention. While complex backgrounds, non-rigid movements of pedestrians and limitations of surveillance cameras and multi-task requirements for a surveillance system all pose challenges to the development of such systems, a further challenge is the detection of crisis behaviours before a suicide attempt is made, and there is a paucity of datasets in this area due to privacy and confidentiality issues. Most relevant research only applies to detecting suicides such as hangings or jumps from bridges, providing no potential for early prevention. In this research, these problems are addressed by proposing a new modular design for an intelligent visual processing pipeline that is capable of pedestrian detection, tracking, pose estimation and recognition of both normal actions and high risk behavioural cues that are important indicators of a suicide attempt. Specifically, based on the key finding that human body gestures can be used for the detection of social signals that potentially precede a suicide attempt, a new 2D skeleton-based action recognition algorithm is proposed. By using a two-branch network that takes advantage of three types of skeleton-based features extracted from a sequence of frames and a stacked LSTM structure, the model predicts the action label at each time step. It achieved good performance on both the public dataset JHMDB and a smaller private CCTV footage collection on action recognition. Moreover, a logical layer, which uses knowledge from a human coding study to recognise pre-suicide behaviour indicators, has been built on top of the action recognition module to compensate for the small dataset size. It enables complex behaviour patterns to be recognised even from smaller datasets. The whole pipeline has been tested in a real-world application of suicide prevention using simulated footage from a surveillance system installed at a suicide hotspot, and preliminary results confirm its effectiveness at capturing crisis behaviour indicators for early detection and prevention of suicide.

Machine learning model to predict mental health crises from electronic health records

The timely identification of patients who are at risk of a mental health crisis can lead to improved outcomes and to the mitigation of burdens and costs. However, the high prevalence of mental health problems means that the manual review of complex patient records to make proactive care decisions is not feasible in practice. Therefore, we developed a machine learning model that uses electronic health records to continuously monitor patients for risk of a mental health crisis over a period of 28 days. The model achieves an area under the receiver operating characteristic curve of 0.797 and an area under the precision-recall curve of 0.159, predicting crises with a sensitivity of 58% at a specificity of 85%. A follow-up 6-month prospective study evaluated our algorithm’s use in clinical practice and observed predictions to be clinically valuable in terms of either managing caseloads or mitigating the risk of crisis in 64% of cases. To our knowledge, this study is the first to continuously predict the risk of a wide range of mental health crises and to explore the added value of such predictions in clinical practice.

Machine learning applied on electronic health records can predict mental health crises 28 days in advance and become a clinically valuable tool for managing caseloads and mitigating the risk of crisis.

Deep cascaded multitask framework for detection of temporal orientation, sentiment and emotion from suicide notes

With the upsurge in suicide rates worldwide, timely identification of the at-risk individuals using computational methods has been a severe challenge. Anyone presenting with suicidal thoughts, mainly recurring and containing a deep desire to die, requires urgent and ongoing psychiatric treatment. This work focuses on investigating the role of temporal orientation and sentiment classification (auxiliary tasks) in jointly analyzing the victims’ emotional state (primary task). Our model leverages the effectiveness of multitask learning by sharing features among the tasks through a novel multi-layer cascaded shared-private attentive network. We conducted our experiments on a diversified version of the prevailing standard emotion annotated corpus of suicide notes in English, CEASE-v2.0. Experiments show that our proposed multitask framework outperforms the existing state-of-the-art system by 3.78% in the Emotion task, with a cross-validation Mean Recall (MR) of 60.90%. From our empirical and qualitative analysis of results, we observe that learning the tasks of temporality and sentiment together has a clear correlation with emotion recognition.

Translating promise into practice: a review of machine learning in suicide research and prevention

In ever more pressured health-care systems, technological solutions offering scalability of care and better resource targeting are appealing. Research on machine learning as a technique for identifying individuals at risk of suicidal ideation, suicide attempts, and death has grown rapidly. This research often places great emphasis on the promise of machine learning for preventing suicide, but overlooks the practical, clinical implementation issues that might preclude delivering on such a promise. In this Review, we synthesise the broad empirical and review literature on electronic health record-based machine learning in suicide research, and focus on matters of crucial importance for implementation of machine learning in clinical practice. The challenge of preventing statistically rare outcomes is well known; progress requires tackling data quality, transparency, and ethical issues. In the future, machine learning models might be explored as methods to enable targeting of interventions to specific individuals depending upon their level of need-ie, for precision medicine. Primarily, however, the promise of machine learning for suicide prevention is limited by the scarcity of high-quality scalable interventions available to individuals identified by machine learning as being at risk of suicide.

Psychosis Relapse Prediction Leveraging Electronic Health Records Data and Natural Language Processing Enrichment Methods

BACKGROUND

Identifying patients at a high risk of psychosis relapse is crucial for early interventions. A relevant psychiatric clinical context is often recorded in clinical notes; however, the utilization of unstructured data remains limited. This study aimed to develop psychosis-relapse prediction models using various types of clinical notes and structured data.

METHODS

Clinical data were extracted from the electronic health records of the Ajou University Medical Center in South Korea. The study population included patients with psychotic disorders, and outcome was psychosis relapse within 1 year. Using only structured data, we developed an initial prediction model, then three natural language processing (NLP)-enriched models using three types of clinical notes (psychological tests, admission notes, and initial nursing assessment) and one complete model. Latent Dirichlet Allocation was used to cluster the clinical context into similar topics. All models applied the least absolute shrinkage and selection operator logistic regression algorithm. We also performed an external validation using another hospital database.

RESULTS

A total of 330 patients were included, and 62 (18.8%) experienced psychosis relapse. Six predictors were used in the initial model and 10 additional topics from Latent Dirichlet Allocation processing were added in the enriched models. The model derived from all notes showed the highest value of the area under the receiver operating characteristic (AUROC = 0.946) in the internal validation, followed by models based on the psychological test notes, admission notes, initial nursing assessments, and structured data only (0.902, 0.855, 0.798, and 0.784, respectively). The external validation was performed using only the initial nursing assessment note, and the AUROC was 0.616.

CONCLUSIONS

We developed prediction models for psychosis relapse using the NLP-enrichment method. Models using clinical notes were more effective than models using only structured data, suggesting the importance of unstructured data in psychosis prediction.

Frequency of Clinicians' Assessments for Access to Lethal Means in Persons at Risk for Suicide

OBJECTIVE

We measured the frequency of clinicians' assessments for access to lethal means, including firearms and medications in patients at risk of suicide from electronic medical and mental health records in outpatient and emergency settings.

METHODS

We included adult patients who reported suicide ideation on the PHQ-9 depression screener in behavioral health and primary care outpatient settings of a large integrated health system in the U.S. and those with suicidal behavior treated in the emergency department. Two separate natural language processing queries were developed on medical record text documentation: (1) assessment for access to firearms (8,994 patients), (2) assessment for access to medications (4,939 patients).

RESULTS

Only 35% of patients had documentation of firearm or medication assessment in the month following treatment for suicidal behavior in the emergency setting. Among those reporting suicidal ideation in outpatient setting, 31% had documentation of firearm assessment and 23% for medication assessment. The accuracy of the estimates was very good for firearm assessment (F1 = 89%) and medication assessment in the outpatient setting (F1 = 91%) and fair for medication assessment in the emergency setting (F1 = 70%) due to more varied documentation styles.

CONCLUSIONS

Lethal means assessment following report of suicidal ideation or behavior is low in a nonacademic health care setting. Until health systems implement more structured documentation to measure lethal means assessment, such as discrete data field, NLP methods may be used to conduct research and surveillance of this important prevention practice in real-world settings.

Using natural language processing to extract self-harm and suicidality data from a clinical sample of patients with eating disorders: a retrospective cohort study

OBJECTIVES

The objective of this study was to determine risk factors for those diagnosed with eating disorders who report self-harm and suicidality.

DESIGN AND SETTING

This study was a retrospective cohort study within a secondary mental health service, South London and Maudsley National Health Service Trust.

PARTICIPANTS

All diagnosed with an F50 diagnosis of eating disorder from January 2009 to September 2019 were included.

INTERVENTION AND MEASURES

Electronic health records (EHRs) for these patients were extracted and two natural language processing tools were used to determine documentation of self-harm and suicidality in their clinical notes. These tools were validated manually for attribute agreement scores within this study.

RESULTS

The attribute agreements for precision of positive mentions of self-harm were 0.96 and for suicidality were 0.80; this demonstrates a 'near perfect' and 'strong' agreement and highlights the reliability of the tools in identifying the EHRs reporting self-harm or suicidality. There were 7434 patients with EHRs available and diagnosed with eating disorders included in the study from the dates January 2007 to September 2019. Of these, 4591 (61.8%) had a mention of self-harm within their records and 4764 (64.0%) had a mention of suicidality; 3899 (52.4%) had mentions of both. Patients reporting either self-harm or suicidality were more likely to have a diagnosis of anorexia nervosa (AN) (self-harm, AN OR=3.44, 95% CI 1.05 to 11.3, p=0.04; suicidality, AN OR=8.20, 95% CI 2.17 to 30.1; p=0.002). They were also more likely to have a diagnosis of borderline personality disorder (p≤0.001), bipolar disorder (p<0.001) or substance misuse disorder (p<0.001).

CONCLUSION

A high percentage of patients (>60%) diagnosed with eating disorders report either self-harm or suicidal thoughts. Relative to other eating disorders, those diagnosed with AN were more likely to report either self-harm or suicidal thoughts. Psychiatric comorbidity, in particular borderline personality disorder and substance misuse, was also associated with an increase risk in self-harm and suicidality. Therefore, risk assessment among patients diagnosed with eating disorders is crucial.

Detection of self-harm and suicidal ideation in emergency department triage notes

OBJECTIVE

Accurate identification of self-harm presentations to Emergency Departments (ED) can lead to more timely mental health support, aid in understanding the burden of suicidal intent in a population, and support impact evaluation of public health initiatives related to suicide prevention. Given lack of manual self-harm reporting in ED, we aim to develop an automated system for the detection of self-harm presentations directly from ED triage notes.

MATERIALS AND METHODS

We frame this as supervised classification using natural language processing (NLP), utilizing a large data set of 477 627 free-text triage notes from ED presentations in 2012-2018 to The Royal Melbourne Hospital, Australia. The data were highly imbalanced, with only 1.4% of triage notes relating to self-harm. We explored various preprocessing techniques, including spelling correction, negation detection, bigram replacement, and clinical concept recognition, and several machine learning methods.

RESULTS

Our results show that machine learning methods dramatically outperform keyword-based methods. We achieved the best results with a calibrated Gradient Boosting model, showing 90% Precision and 90% Recall (PR-AUC 0.87) on blind test data. Prospective validation of the model achieves similar results (88% Precision; 89% Recall).

DISCUSSION

ED notes are noisy texts, and simple token-based models work best. Negation detection and concept recognition did not change the results while bigram replacement significantly impaired model performance.

CONCLUSION

This first NLP-based classifier for self-harm in ED notes has practical value for identifying patients who would benefit from mental health follow-up in ED, and for supporting surveillance of self-harm and suicide prevention efforts in the population.

The Language of the Times: Using the COVID-19 Pandemic to Assess the Influence of News Affect on Online Mental Health-Related Search Behavior across the United States

Background

The digital era has ushered in an unprecedented volume of readily accessible information, including news coverage of current events. Research has shown that the sentiment of news articles can evoke emotional responses from readers on a daily basis with specific evidence for increased anxiety and depression in response to coverage of the recent COVID-19 pandemic. Given the primacy and relevance of such information exposure, its daily impact on the mental health of the general population within this modality warrants further nuanced investigation.

Objective

Using the COVID-19 pandemic as a subject-specific example, this work aimed to profile and examine associations between the dynamics of semantic affect in online local news headlines and same-day online mental health term search behavior over time across the United States.

Methods

Using COVID-19–related news headlines from a database of online news stories in conjunction with mental health–related online search data from Google Trends, this paper first explored the statistical and qualitative affective properties of state-specific COVID-19 news coverage across the United States from January 23, 2020, to October 22, 2020. The resultant operationalizations and findings from the joint application of dictionary-based sentiment analysis and the circumplex theory of affect informed the construction of subsequent hypothesis-driven mixed effects models. Daily state-specific counts of mental health search queries were regressed on circumplex-derived features of semantic affect, time, and state (as a random effect) to model the associations between the dynamics of news affect and search behavior throughout the pandemic. Search terms were also grouped into depression symptoms, anxiety symptoms, and nonspecific depression and anxiety symptoms to model the broad impact of news coverage on mental health.

Results

Exploratory efforts revealed patterns in day-to-day news headline affect variation across the first 9 months of the pandemic. In addition, circumplex mapping of the most frequently used words in state-specific headlines uncovered time-agnostic similarities and differences across the United States, including the ubiquitous use of negatively valenced and strongly arousing language. Subsequent mixed effects modeling implicated increased consistency in affective tone (Spin_VA β=–.207; P<.001) as predictive of increased depression-related search term activity, with emotional language patterns indicative of affective uncontrollability (Flux_A β=.221; P<.001) contributing generally to an increase in online mental health search term frequency.

Conclusions

This study demonstrated promise in applying the circumplex model of affect to written content and provided a practical example for how circumplex theory can be integrated with sentiment analysis techniques to interrogate mental health–related associations. The findings from pandemic-specific news headlines highlighted arousal, flux, and spin as potentially significant affect-based foci for further study. Future efforts may also benefit from more expansive sentiment analysis approaches to more broadly test the practical application and theoretical capabilities of the circumplex model of affect on text-based data.

Digital conversations about depression among Hispanics and non-Hispanics in the US: a big-data, machine learning analysis identifies specific characteristics of depression narratives in Hispanics

BACKGROUND

Digital conversations can offer unique information into the attitudes of Hispanics with depression outside of formal clinical settings and help generate useful information for medical treatment planning. Our study aimed to explore the big data from open-source digital conversations among Hispanics with regard to depression, specifically attitudes toward depression comparing Hispanics and non-Hispanics using machine learning technology.

METHODS

Advanced machine-learning empowered methodology was used to mine and structure open-source digital conversations of self-identifying Hispanics and non-Hispanics who endorsed suffering from depression and engaged in conversation about their tone, topics, and attitude towards depression. The search was limited to 12 months originating from US internet protocol (IP) addresses. In this cross-sectional study, only unique posts were included in the analysis and were primarily analyzed for their tone, topic, and attitude towards depression between the two groups using descriptive statistical tools.

RESULTS

A total of 441,000 unique conversations about depression, including 43,000 (9.8%) for Hispanics, were posted. Source analysis revealed that 48% of conversations originated from topical sites compared to 16% on social media. Several critical differences were noted between Hispanics and non-Hispanics. In a higher percentage of Hispanics, their conversations portray "negative tone" due to depression (66% vs 39% non-Hispanics), show a resigned/hopeless attitude (44% vs. 30%) and were about 'living with' depression (44% vs. 25%). There were important differences in the author's determined sentiments behind the conversations among Hispanics and non-Hispanics.

CONCLUSION

In this first of its kind big data analysis of nearly a half-million digital conversations about depression using machine learning, we found that Hispanics engage in an online conversation about negative, resigned, and hopeless attitude towards depression more often than non-Hispanic.

Morphological characteristics of spoken language in schizophrenia patients - an exploratory study

Psychosis is diagnosed based on disruptions in the structure and use of language, including reduced syntactic complexity, derailment, and tangentiality. With the development of computational analysis, natural language processing (NLP) techniques are used in many areas of life to make evaluations and inferences regarding people's thoughts, feelings and behavior. The present study explores morphological characteristic of schizophrenia inpatients using NLP. Transcripts of recorded stories by 49 male subjects (24 inpatients diagnosed with schizophrenia and 25 controls) about 14 Thematic Apperception Test (TAT) pictures were morphologically analyzed. Relative to the control group, the schizophrenic inpatients employed: (1) a similar ratio of nouns, but fewer verbs, adjectives and adverbs; (2) a higher ratio of lemmas to token (LTR) and type to token (TTR); (3) a smaller gap between LTR and TTR; and (4) greater use of the first person. The results were cross-verified using three well-known fitting classifier algorithms (Random Forest, XGBoost and a support vector machine). Tests of prediction accuracy, precision and recall found correct attribution of patients to the schizophrenia group at a rate of between 80 and 90%. Overall, the results suggest that the language of schizophrenic inpatients is significantly different from that of healthy controls, being morphologically less complex, more associative and more focused on the self. The findings support NLP analysis as a complementary addition to the traditional clinical psychosis evaluation for schizophrenia.

Identifying themes for assessing cancer-related cognitive impairment identified by topic modeling and qualitative content analysis of public online comments (Preprint)

Background

Cancer-related cognitive impairment (CRCI) is a common and significant adverse effect of cancer and its therapies. However, its definition and assessment remain difficult due to limitations of currently available measurement tools.

Objective

This study aims to evaluate qualitative themes related to the cognitive effects of cancer to help guide development of assessments that are more specific than what is currently available.

Methods

We applied topic modeling and inductive qualitative content analysis to 145 public online comments related to cognitive effects of cancer.

Results

Topic modeling revealed 2 latent topics that we interpreted as representing internal and external factors related to cognitive effects. These findings lead us to hypothesize regarding the potential contribution of locus of control to CRCI. Content analysis suggested several major themes including symptoms, emotional/psychological impacts, coping, “chemobrain” is real, change over time, and function. There was some conceptual overlap between the 2 methods regarding internal and external factors related to patient experiences of cognitive effects.

Conclusions

Our findings indicate that coping mechanisms and locus of control may be important themes to include in assessments of CRCI. Future directions in this field include prospective acquisition of free-text responses to guide development of assessments that are more sensitive and specific to cognitive function in patients with cancer.

Developing a Natural Language Processing tool to identify perinatal self-harm in electronic healthcare records

BACKGROUND

Self-harm occurring within pregnancy and the postnatal year ("perinatal self-harm") is a clinically important yet under-researched topic. Current research likely under-estimates prevalence due to methodological limitations. Electronic healthcare records (EHRs) provide a source of clinically rich data on perinatal self-harm.

AIMS

(1) To create a Natural Language Processing (NLP) tool that can, with acceptable precision and recall, identify mentions of acts of perinatal self-harm within EHRs. (2) To use this tool to identify service-users who have self-harmed perinatally, based on their EHRs.

METHODS

We used the Clinical Record Interactive Search system to extract de-identified EHRs of secondary mental healthcare service-users at South London and Maudsley NHS Foundation Trust. We developed a tool that applied several layers of linguistic processing based on the spaCy NLP library for Python. We evaluated mention-level performance in the following domains: span, status, temporality and polarity. Evaluation was done against a manually coded reference standard. Mention-level performance was reported as precision, recall, F-score and Cohen's kappa for each domain. Performance was also assessed at 'service-user' level and explored whether a heuristic rule improved this. We report per-class statistics for service-user performance, as well as likelihood ratios and post-test probabilities.

RESULTS

Mention-level performance: micro-averaged F-score, precision and recall for span, polarity and temporality >0.8. Kappa for status 0.68, temporality 0.62, polarity 0.91. Service-user level performance with heuristic: F-score, precision, recall of minority class 0.69, macro-averaged F-score 0.81, positive LR 9.4 (4.8-19), post-test probability 69.0% (53-82%). Considering the task difficulty, the tool performs well, although temporality was the attribute with the lowest level of annotator agreement.

CONCLUSIONS

It is feasible to develop an NLP tool that identifies, with acceptable validity, mentions of perinatal self-harm within EHRs, although with limitations regarding temporality. Using a heuristic rule, it can also function at a service-user-level.

Harm minimisation for the management of self-harm: a mixed-methods analysis of electronic health records in secondary mental healthcare

BACKGROUND

Prevalence of self-harm in the UK was reported as 6.4% in 2014. Despite sparse evidence for effectiveness, guidelines recommend harm minimisation; a strategy in which people who self-harm are supported to do so safely.

AIMS

To determine the prevalence, sociodemographic and clinical characteristics of those who self-harm and practise harm minimisation within a London mental health trust.

METHOD

We included electronic health records for patients treated by South London and Maudsley NHS Trust. Using an iterative search strategy, we identified patients who practise harm minimisation, then classified the approaches using a content analysis. We compared the sociodemographic characteristics with that of a control group of patients who self-harm and do not use harm minimisation.

RESULTS

In total 22 736 patients reported self-harm, of these 693 (3%) had records reporting the use of harm-minimisation techniques. We coded the approaches into categories: (a) 'substitution' (>50% of those using harm minimisation), such as using rubber bands or using ice; (b) 'simulation' (9%) such as using red pens; (c) 'defer or avoid' (7%) such as an alternative self-injury location; (d) 'damage limitation' (9%) such as using antiseptic techniques; the remainder were unclassifiable (24%). The majority of people using harm minimisation described it as helpful (>90%). Those practising harm minimisation were younger, female, of White ethnicity, had previous admissions and were less likely to have self-harmed with suicidal intent.

CONCLUSIONS

A small minority of patients who self-harm report using harm minimisation, primarily substitution techniques, and the large majority find harm minimisation helpful. More research is required to determine the acceptability and effectiveness of harm-minimisation techniques and update national clinical guidelines.

Natural language processing of clinical mental health notes may add predictive value to existing suicide risk models

BACKGROUND

This study evaluated whether natural language processing (NLP) of psychotherapy note text provides additional accuracy over and above currently used suicide prediction models.

METHODS

We used a cohort of Veterans Health Administration (VHA) users diagnosed with post-traumatic stress disorder (PTSD) between 2004-2013. Using a case-control design, cases (those that died by suicide during the year following diagnosis) were matched to controls (those that remained alive). After selecting conditional matches based on having shared mental health providers, we chose controls using a 5:1 nearest-neighbor propensity match based on the VHA's structured Electronic Medical Records (EMR)-based suicide prediction model. For cases, psychotherapist notes were collected from diagnosis until death. For controls, psychotherapist notes were collected from diagnosis until matched case's date of death. After ensuring similar numbers of notes, the final sample included 246 cases and 986 controls. Notes were analyzed using Sentiment Analysis and Cognition Engine, a Python-based NLP package. The output was evaluated using machine-learning algorithms. The area under the curve (AUC) was calculated to determine models' predictive accuracy.

RESULTS

NLP derived variables offered small but significant predictive improvement (AUC = 0.58) for patients that had longer treatment duration. A small sample size limited predictive accuracy.

CONCLUSIONS

Study identifies a novel method for measuring suicide risk over time and potentially categorizing patient subgroups with distinct risk sensitivities. Findings suggest leveraging NLP derived variables from psychotherapy notes offers an additional predictive value over and above the VHA's state-of-the-art structured EMR-based suicide prediction model. Replication with a larger non-PTSD specific sample is required.

Applications of Aspect-based Sentiment Analysis on Psychiatric Clinical Notes to Study Suicide in Youth

Understanding and identifying the risk factors associated with suicide in youth experiencing mental health concerns is paramount to early intervention. 45% of patients are admitted annually for suicidality at BC Children's Hospital. Natural Language Processing (NLP) approaches have been applied with moderate success to psychiatric clinical notes to predict suicidality. Our objective was to explore whether machine-learning-based sentiment analysis could be informative in such a prediction task. We developed a psychiatry-relevant lexicon and identified specific categories of words, such as thought content and thought process that had significantly different polarity between suicidal and non-suicidal cases. In addition, we demonstrated that the individual words with their associated polarity can be used as features in classification models and carry informative content to differentiate between suicidal and non-suicidal cases. In conclusion, our study reveals that there is much value in applying NLP to psychiatric clinical notes and suicidal prediction.